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Preface 



In the field of control systems, one area which has been studied extensively in the 
academic environment this past decade has encountered however as far as now 
relatively low acceptance in the industrial world. It is the area of control of multi- 
ple input plants, or of estimation of the states of multiple output plants. One ex- 
ception is the decoupling control, but here also merely one or two design methods 
have found real applications. On another hand, though or maybe even because of 
the importance of digital control in practical realizations, it is essential to be at 
ease with these control or estimation methods, both in continuous and in discrete 
time, so as to be able to simulate them on a computer in the two forms. It is com- 
mon practice, indeed, to describe an industrial process or plant initially by a con- 
tinuous model, then to design and test adequate control schemes in continuous 
time, before switching to discrete time for the final computer implementation. 

The book essentially consists of three parts. The first part, composed of Chap- 
ters 1 and 2, deals with the concepts of linear control laws and observers. A rela- 
tively unknown generalizing formula for multivariable controller or observer de- 
sign, which encompasses some of the other methods while removing certain of 
their limitations, is also given. The basic prerequisites for the reader are here 
ground knowledge in Matrix Calculus and Linear Algebra. Concise reviews are 
given in the Appendices on these topics and the chapters are written with a clear 
mathematical approach, with proofs, so that the student as well as the engineer can 
follow the developments completely, without unnecessary details. The second part 
of the book, covering Chapters 3 to 5, deals with quadratic methods to determine 
control or estimation laws. The basics required to follow these chapters are essen- 
tially the same as before, except for Chapter 4 which requires some knowledge of 
Probability Calculus, the main concepts of which are also summarized in an Ap- 
pendix. The third part of the book is represented by a single chapter which de- 
scribes a rather new design method, valid as well for controllers as for observers, 
relatively unknown to the best of our knowledge from the industrial world: the 
method based on Linear Matrix Inequalities. This approach is presented here in a 
simple but rigorous way, so that no special prerequisite is needed to understand it. 
The reader might be surprised to discover how easily problems of the linear part as 
well as of the quadratic part can be translated into this elegant formalism, and how 
easily they can then be solved by available, freely downloadable software. 
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In all chapters, the continuous-time and discrete-time situations are treated al- 
most always in parallel. The description tool used for the plants and for the mathe- 
matical developments is essentially the state space representation, sometimes also 
called internal representation, which is particularly suitable for multivariable sys- 
tems. Many new methods for such plants are simply not tractable with other repre- 
sentations. Their use in this book with single input or single output systems, to 
which they apply evidently also, serves essentially to establish the basic theorems 
and their proofs in a highly readable way, before generalizing to the multivariable 
case. 

Many exercises are given at the end of each chapter, with their complete solu- 
tions, most of them obtained in the MATLAB® environment. The programs used 
for their solving are grouped in an archive, which is downloadable from either the 
Springer or the Math Works Internet site. The user can modify them at will by add- 
ing other solutions or other plant models of his own. These exercises represent in 
our mind the most important part of the learning process, and the reader is 
strongly encouraged to try the solutions by himself before reading them in the 
book. The simulations of these exercises have been realized by three Simulink® 
diagrams, given in one of the Appendices and also included in the downloadable 
archive for the reader's use. 

This book is aimed at several kinds of public. It addresses first students in En- 
gineering Schools or Universities, at the Master's level. It is also aimed at Control 
Engineers, faced with the control of industrial processes, where new specifications 
reaching far beyond the possibilities of the classical PID controller are imposed. 
Finally, of course, this book is also written for researchers wanting to broaden 
their knowledge of control approaches, in particular in the multivariable case. 

Among the persons who contributed to the present shape of this book, the au- 
thor is grateful to Maria Joana Carvalho, Assistant Professor at the Ecole Nation- 
ale Superieure de Physique of the University of Strasbourg, for her helpful com- 
ments and suggestions, and to Gilles Due, Professor at Supelec (formerly Ecole 
Superieure d'Electricite), for his careful reading and valuable suggestions of im- 
provement in several parts of the book. I would like to express here also my grati- 
tude to Gunter Roppenecker, Professor at the University Erlangen-Nurnberg, for 
his kind support in clarifying the design method bearing his name. 

The author is also deeply indebted to Eva Hestermann-Beyerle, Senior Engi- 
neering Editor at Springer, for her kind invitation to publish this book in this new 
Springer series and helpful advices. 

Suggestions of improvements and corrections about the book itself or the ac- 
companying software will be of course greatly appreciated and can be sent to 
Eric.Ostertag@unistra.fr. 

Summer 2010 Eric Ostertag 
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NOTATIONAL CONVENTIONS 



• Scalar variables and scalar functions are denoted by lowercase Latin and Greek 
letters. 

Examples: y(t) e 

Exception: T s , sampling time of discrete-time signals 

• For discrete -time functions, the sample instant is indicated as a subscript, rather 
than in parentheses, in order to simplify the notations. 

Examples: y k+1 , and not y(k + 1) 

• Vectors are denoted by lowercase bold Latin and Greek letters. 

Examples: u k y 

• Matrices are denoted by capital bold Latin and Greek letters. 

Examples: A O 

• Transforms (Laplace, z) of scalar or vectorial functions use the corresponding 
uppercase Latin and Greek letters: 



Time function 


Transform 


x(t) , x(0 


C[x(t)} = X(s),C[x(t)} = X(s) 


J*, y* 


£[y k } = Y(z),£[y k } = Y(z) 


m symbol: superscript T 




pies: A T c T 





Special notations for stochastic environments: 

- scalar or vectorial random variables and scalar or vectorial random func- 
tions are denoted by capital Latin letters. 

Examples: X(t) , Y k , V(t) , W t 

- deterministic realizations of such random variables or functions are de- 
noted by the corresponding lowercase letter. 



Examples: x(t) , y k , \(t) , w 



k 



N.B.: there should be no confusion with the above convention concerning the 
use of capital letters, for Laplace or z transforms and for matrices. In the first case, 
the dependent variable, s or z, will always be given explicitly. In the second case, 
no confusion should arise, since no random matrices will appear in this book. 



/ Notations 

Estimates symbol: the upper hat symbol ( ) 

Examples: x x(t) x k \ k 

Estimation errors symbol: the upper tilde symbol ( ~ ) 
Examples: x x(t) x k 

This symbol is also used, without risk of ambiguity, for matrices and vectors of 

augmented systems: 

Examples: A L 

Similitude transformations, or basis changes: matrix and vectors resulting from 
such changes are denoted by a circumflex accent. 

Examples: A , v 
Linear systems: 

- state vector: x(t) or x^ £ W (real vectors, of dimensions) 

- input: u(t) or u k £R P (real vectors, of dimension/?) 

- output: y(t) or y k gR' (real vectors, of dimension q) 
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For all questions relating to the definitions and properties of the state-space repre- 
sentations of linear time-invariant (LTI) systems, and to the notations used in this 
book, the reader is invited to consult Appendix A at the end of the book as well as 
the page on Conventional Notations at its beginning. 

A deliberate choice has been made not to describe in this book how these state 
space models are established, nor to tackle the numerous other modelization or 
model reduction methods existing for such systems. The reader may consult for 
this purpose various text books, such as [AlSa04], [Lev96], [Oga02], [ZhDG96]. 

To underline better the resemblance of many methods and formulae used in the 
continuous case and in the discrete case, this chapter and the following ones will 
deal simultaneously with both system types. The particulars of each type (formu- 
lae or proofs, e.g.) will be mentioned and developed, if necessary, in parallel. The 
two types of systems are distinguished all along this book by the different nota- 
tions involved in their state representations. 

Care has been taken also to identify differently in their notations the monovari- 
able systems, including systems with only one input or one output or both, the so- 
called single input - single output (SISO) systems, and the multivariable systems, 
commonly designated in the control literature by multiple input - multiple output 
(MIMO) systems. These two abbreviations will be used throughout this manual. 



1.1 State Space Control: an Introduction 



1.1.1 Problem Description 

We shall suppose in all the following that the process to be controlled, hereafter 
called the plant or the open-loop system, is a linear time-invariant system. Fur- 
thermore, to simplify the notations, only systems for which D = will be consid- 
ered here, which is the great majority of real systems (see Sect. A. 1.1 and A. 1.2). 
The plant will thus be described by one of the following state representations: 
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Continuous Case: 
x(0 = Ax(0 + Bu(0 or: 
y(0 = Cx(0 
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Discrete Case: 



y* 



= <&x t + ru t 

= Cx t 



with: AorO: system matrix, constant, real, of size (nxn), i.e. <E M" x " ; 

x: state vector, real, of dimension n, i.e. € K" ; 

u: input vector, real, of dimension p <n , i.e. € R p ; 

B or r : input matrix, constant, real, of size (n x p) , i.e. 6R" xp ; 

y: output or measurement vector, real, of dimension q <n , i.e. € M ? ; 

C: output or measurement matrix, constant, real, of size (qxn), € M ?x " . 
The system being at initial time t Q ( = for time-invariant systems) in the ini- 
tial state x , the aim is to transfer it to the desired final state, or operating state, 
Xy , and this, while submitting the transient phase to given dynamic requirements, 
such as rise time or transient damping. Once the operating state x f is reached, the 
output quantity y must usually be equal to an imposed reference value y r . 



1.1.2 Solution: State Feedback 

The way the problem is solved consists in feeding back the state vector x to the 
input by means of a feedback matrix L, as shown in Fig. 1 . 1 in the continuous 
case, x is supposed here to be fully accessible, a restriction which will be recon- 
sidered in Sect. 1.1.5 (Remark 1.3). Such a control law is called state feedback. 



1.1.3 Continuous Case 



Plant (or Open-loop System) x o 

,+ 




Static Gain 

Compensation 

(or Feedforward) 

Matrix 

(pxq) 



Controller (pxn) 
Fig. 1.1 State feedback control of a continuous time plant. 
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L is a constant (p x n) matrix called state controller. Its practical realization 
will consist of a set of constant elements, thus of proportional terms. It appears 
thus to play the role of a proportional controller, having in the extreme case of n 
elements. Note however that the quantities calculated this way are proportional not 
only to the output but also to all its derivatives, as shown e.g. by (A.l) where 
x ; = x i+l . The state-feedback controller is thus rather equivalent to a PD control- 
ler. However it does not contain any "I" -term (integrator), point which will be ad- 
dressed in Sect. 1.8. The role of the M matrix will be explained in Sect. 1.1.3.2. 

The equations of the system shown in Fig. 1.1 are the following: 

x = Ax + Bu; y = Cx; (1.1) 

u = u L +u M =-Lx + My r . (1.2) 

Note that the time dependency of the time-varying elements of these equations has 
been omitted. This will be the case throughout this book for continuous-time sys- 
tems. The above equations lead to the following closed-loop state equation: 

x = (A-BL)x + BMy r . (1.3) 

The closed-loop system is thus described by a state equation, with system matrix 

A CL =A BL, (1.4) 

and input matrix 

B Ci = BM. (1.5) 

Since two types of external quantities act on this control system, the initial state 
x and the reference signal y r , two kinds of behavior can be considered. 



1.1.3.1 Regulation Behavior 

The regulation methods, which will be described in the following sections, aim at 
transferring the system state vector from the initial state x to the operating (final) 

state x ,■ under given conditions. This will result from an adequate choice of L. 

The system being linear, it is then possible, in order to calculate L, to set 
y r = , according to the linear superposition principle. Equation (1.3) simplifies 

then to x = (A — BL) x . Once the equilibrium is reached (steady state), x = . It 

results necessarily in this case, provided all closed-loop eigenvalues have been 
chosen in the left-half s plane, thus that A BL is invertible, that 



= 0. 



1 State Space Control Design 
(1.6) 



1.1.3.2 Servo Behavior: Choice of Gain Compensation (or 
Feedforward) Matrix 

In the steady state, which corresponds to x = , the equality y = y r must hold. In 
this state, (1.3) is written 



= (A-BL)x + BMy r , 



therefore: 



x = (BL A)'BMy r . 
The output equation yields then 

y = y,=C(BL A)'BMy r , 



from where follows that 



C(BL-A)"'BM = I. 



(1.7) 



Case where p — q: this case is often encountered in practice, where one has gener- 
ally to associate with each output signal a reference signal, which is expected to 
influence as wanted this output. Such plants are called square plants. M is then a 
square, thus possibly invertible, matrix, and (1.7) yields 



(1.8) 



Remark 1.1. Not surprisingly, the M matrix is nothing else than the inverse of the 
closed-loop static gain. This gain is obtained, indeed, by letting s = in the ex- 
pression of the transfer matrix of a system in state-space representation, namely, 
for the closed-loop system with the assumption D = (see Appendix A, Sect. 

A.5), G a (5) = C(5l-A ci )- 1 B: 



M= C(BL A) "B 



G a (s)|, =0 =C[*I-(A-BL) 



B 



C(BL -A)"'B. 



(1.9) 



Remark 1.2. The static gain compensation realized by the M matrix is ideal only 
if the plant parameters, on which the value of M depends, are perfectly known and 
do not vary with time. To compensate for these two restrictions, it may be advan- 
tageous to add an integral term to the control loop, as will be seen in Sect. 1.8. 
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Case where p^q, most often p<q : since the matrix in the second member of 

(1.9) is no longer square, thus not invertible, M cannot be calculated any more by 
(1.8). It is then possible to apply the unity static gain requirement only between a 
number of outputs equal to that of available control inputs and their corresponding 
reference values. This amounts to choosing a submatrix of C which has only p 
rows, which brings back to the previous case (p — q) . This is the purpose of the 

selection matrix S c introduced in the simulation diagrams of Appendix D. For 
systems with p > q one will have, on the contrary, to drop p — q control inputs 
of the plant for that requirement. 



1.1.4 Discrete Case 

Let us represent this time the plant in its general form by a state-variable flow 
graph, where T s is the sampling time (see Fig. 1.2): 



Plant (or Open-loop System) 




Static Gain 

Compensation 

(or Feedforward) 

Matrix 

(pXq) 

Controller (pXn) 

Fig. 1.2 State feedback control of a discrete- time plant. 

The corresponding equations are here 



(1.10) 

(l.ii) 

(1.12) 



\ k is assumed here again totally accessible. The closed-loop state equation is de- 
rived by substituting (1.12) into (1.11): 

x t+1 =(0-rL)x t +rMy rit . (1.13) 

The closed-loop system is again described by a state equation, with system matrix 
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CL =0-TL, (1.14) 
and input matrix: 

r Ci =TM. (1.15) 

1.1.4.1 Regulation Behavior 

As in the continuous case, let us set y rk = 0, V£ . The design problem will then 
amount to determine a sequence of control values u , Uj , ■ • • , u k , obtained from 
\ k by u^ = -Lx t , which drives the system from an arbitrary initial state x to 
the origin (xy =0). 

1.1.4.2 Servo Behavior 

In the steady state, which corresponds here to x t+1 = x^ , it is still required to have 
y* —Yrk- Equation (1.13) becomes 



Therefore 
and 

yielding 
and, if p = q : 



x t =(0-rL)x i+ rMy ri 



x^a + TL-Or'rMy^, 



y^Ca + rL-Or'rMy^y^, 



c(nrL-or'rM = i, 



M = fc(i + rL-or 1 r] ' . (i.i6) 



N.B.: the M matrix is here again equal to the inverse of the static gain of the 
closed-loop system, gain which follows from the formula giving the transfer ma- 
trix of a discrete-time system of system matrix O and direct feedthrough matrix 



D = 0, namely G(z) = C(zI — O) T , applied to the closed-loop system with 
the substitution z = 1 : 
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G CL ( z )\ z=l =c[ z i-(&-ri,)}- l r 



= c[i + tl o] ! r. 

z = 1 



1.1.5 Design Methods of the Feedback Matrix L 

In the following sections, two large classes of design methods of the L matrix will 
be described. 



• 



The so-called Pole Placement Method, or Pole Assignment Method, where the 
closed-loop poles are given in advance (they are placed in the s or z plane) and 
where L is determined such that the closed loop gets really these poles. Strictly 
speaking, the values imposed are the eigenvalues of the continuous or discrete 
system matrix of the closed-loop system. The language abuse stems from the 
fact that the poles of the closed-loop system are contained among its eigenval- 
ues (Appendix A, Sect. A. 6). 

This method is well adapted to continuous or discrete systems of SISO or 
MIMO type, but it does not make any use of the additional degree of freedom 
available for the latter type, as will be seen later: as a matter of fact it does not 
use any of the possibilities offered by the eigenstructure of the closed loop sys- 
tem. 

The Finite Settling-Time Design [Kal60], [MuBa64], represents a particular 
case of the previous design method, exclusively reserved to the discrete sys- 
tems. It consists in placing all the poles of the closed-loop discrete transfer 
function at the origin of the complex plane (z -plane). It is the transcription in 
the state-variable domain of the design method of a minimum settling-time 
RST-controller, [AsWi97] [GoOs03]. 

As a matter of fact, this method goes far beyond the scope of this frame, be- 
cause it applies also to nonlinear systems, for which one cannot speak anymore 
of pole placement. Incidentally, the approach for the derivation of the corre- 
sponding algorithm, to be described in Sect. 1.3, will depart entirely from the 
pole placement technique. 

The design methods specific to MIMO systems, which still assign the eigen- 
values of the closed-loop system, but address also its structure, more precisely 
its eigenvectors or the decoupling of certain quantities. 

- The Simple Modal Control or Modal Decoupling [Ros62] is a pole place- 
ment in diagonal representation, but limited to a number of poles not ex- 
ceeding the number of process inputs. 

- The Input-Output Decoupling Method permits in some cases to put the 
closed-loop system in the form of several SISO systems connected in par- 
allel. 
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- The Complete Modal Control extends the modal control approach to the 
whole set of closed-loop eigenvalues, by introducing invariant parameter 
vectors, and encompasses as special cases the two previous methods. 

- Finally, a General Formula for the synthesis of both MIMO controllers 
and observers is presented, which removes some of the restrictions of the 
complete modal design. 

Remark 1.3. It was assumed until now that the state vector x was completely ac- 
cessible. In fact, the design will occur in two steps: 

1. x is assumed accessible, and the state controller L of the control law u = — Lx 

is designed according to one of the mentioned methods; 

2. if x is not fully accessible or not accessible at all, one will then proceed with 
the design of a state reconstructor, the purpose of which will be to yield the 
best possible estimation \(t), respectively \ k , of the state vector from what is 
available to measurement, i.e. from the measurement (or output) vector y(t) , 
respectively y k , and the control vector u(t) , respectively u k . 

In the pure deterministic situation, such a state reconstructor will be called an 
observer, e.g. a Luenberger observer, the synthesis of which will be addressed in 
Chapter 2 of this book. 

In the stochastic situation, i.e. in the presence of noisy signals, such a state re- 
constructor will be an estimating filter or a predictive filter, e.g. a Kalman filter, 
the most famous of this type of reconstructors, the synthesis of which will be ad- 
dressed in Chapter 4 of this book, entitled Optimal Linear Filtering. 

Remark 1.4. There are other approaches for the design of a state-feedback control 
law, totally different from the ones mentioned here above. 

One of them consists in looking for a control law which minimizes an algebraic 
criterion, or cost function, usually quadratic, without any concern about the result- 
ing places of the closed-loop poles: it is called Optimal Control. Due to its impor- 
tance and its distinctive characteristics, this design method will be the subject of a 
separate chapter, Chapter 3 of this book. The duality existing between the formal- 
ism of that approach and the one used in optimal filtering will be presented at the 
end of Chapter 4. 

One more method is the Generalized Predictive Control, or GPC [ClMo87], 
[BoDu96], where it is searched to minimize on a finite horizon a cost function, 
which involves both the future, or predicted, response of the process to present 
and future control signals, supposed known, and the resulting control signals. We 
will not go into this subject in the frame of this book. 

Still another class of methods is that of Robust Control, among which should 
be cited the most known, the H^ optimization method. They will not either be 

the subject of this book. Robustness of optimal control loops and its loss due to the 
inclusion of an estimator into the loop will nevertheless be discussed in Chap. 3 
and 5, respectively. 
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1.2 Pole Placement Method 
1.2.1 Basics of the Method 

Consider the system in Fig. 1.3. Since now only the regulation behavior is of con- 
cern, thus the reaction of the system to the initial condition x , we can let y r = . 
Furthermore, the output equation y = Cx will be ignored, since it has already 
been taken into account at the time of designing the gain compensation term M. 
We deal here only with the regulation of the state vector x. 




(pxn) 
Fig. 1.3 Regulation diagram of a state feedback controlled system 



The design by pole placement method consists in choosing the constant state- 
feedback matrix L such that the eigenvalues \ n , \ L2 , ■ ■ ■ , \ Ln of the closed-loop 

system, represented above in the continuous and the discrete cases, are situated at 
given locations in the complex s or z plane. Since, most of the time, these eigen- 
values are identical to the closed-loop poles, as already recalled previously, the 
terra poles will often be used for them by language abuse. 

Conditions of existence of the matrix L. The question arises now about the exis- 
tence of such a matrix L. [Won67] has shown that, for all the eigenvalues of a sys- 
tem to be displaceable to new arbitrary values X n , X L2 , ■ ■ ■ , \ Ln by means of state 
feedback, it is necessary and sufficient that this system be (completely) controlla- 
ble. For the mathematical controllability, or, in case of uncertain or bad condi- 
tioned plant matrices, practical controllability criterions, see Appendix A. 

So what happens, if the plant is not completely controllable? If this is the case, 
it is possible, by means of a change of state variables x = Tz, to transform it to 

the controllability canonical decomposition (Sect. A.2.3, Appendix A), where z c 
(zg- ) are the controllable (uncontrollable) parts of the new state vector z: 



10 
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"c ) 







c ) 



"O 



Applying now a state feedback control as given by (1.2), i.e., 

■My r , 



Li L 2 



l z c 



yields the following closed loop system: 



\ z O 



A C~ B C L 1 A 12~ B C L 2 

JL 



\ z O 



B C M 



y r - 



The closed-loop system matrix being block-diagonal, its eigenvalues are the 
union of the eigenvalues of A c — B c Lj and those of A^ . The choice of the 
closed-loop eigenvalues is thus not completely arbitrary, since it must contain the 
eigenvalues of A^ . These eigenvalues represent the dynamic behavior of the un- 
controllable part of the plant, which is not modified by the feedback. 

Choice of the poles: 

This choice is guided by practical considerations. If a second order type behav- 
ior is desired for the closed loop, one could choose for instance one dominant 
complex conjugated pole pair plus one real pole of multiplicity order (n — 2) , 

where n is the system order. The choice of one unique real pole of multiplicity or- 
der n is of course also possible. 

The exact location of these poles will be deduced from the dynamic behavior 
desired for the controlled system. The closed-loop state equation is, according to 
(1.3) with y r =0: 



i=(A BL)x. 



(1.17) 



The eigenvalues \ u , A i2 , ■ • ■ , \„ of the closed-loop system matrix A — BL , 
which are now supposed chosen, are the zeros of the closed-loop characteristic 
polynomial 



det [si - ( A - BL)] = s" + a„_, (L) • s n 



•a (L), 



(1.1* 



where the notation a,(L) is used simply to recall that the coefficients of this poly- 
nomial depend on the elements of the unknown matrix L. Choosing arbitrarily 
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these eigenvalues amounts to choosing arbitrarily the characteristic polynomial 
coefficients, since 

p(s) = (s-\ Ll )(s-\ L2 )---(s-\ Ln ) = s n +p n _ l s n ~ 1 -\ \-piS + p . 

By coefficient identification, the following equations are obtained: 

«o( L ) =Po 

a, (L) = », 

. , (1.19) 

a„_ 1 (L) = p„_ l 

i.e. a set of n equations in p x n unknowns, the elements of the L matrix. 

Consequences: 

• the solution is determined if p = 1 (single-input systems); 

• it is undetermined (underdetermined set) if p > 1 (multiple-input systems). 

1.2.2 Single-input Systems 

In this case (p — 1) , the L matrix is reduced to one unique row 1 , 

L = £ J ={£ 1 £ 2 - £„), (1.20) 

and the control law, which is scalar here, becomes according to (1.2): 

u = -£ J \. (1.21) 

As long as n does not exceed 3 or 4, or if the closed-loop system matrix is suffi- 
ciently empty, the system of equations (1.19) can be solved directly to get £ T . 

If on the contrary the plant to be controlled is of a higher order, it is better to 
use the following design method, which will be first described hereafter in the 
case of a plant in controllability canonical representation, and transposed then to 
the case of an arbitrary representation. 



1 By exception with the typographic convention adopted in this book, the symbol £ will be 
used here instead of 1 , in order to avoid confusion with the identity matrix I . 
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1.2.2.1 Plants in Controllability Canonical Form 

If the plant is given in controllability canonical form, the closed-loop system ma- 
trix is, according to (1.3) or (1.17), joined to (1.20), 



A CL= A C- b C i C> 



(1.22) 



where A c and b c are given by (A. 1) or (A.31) of Appendix A, recalled here: 



Clf\ t/i 



*n-\) 



This yields 



A Ci = 



10- 







'Q- 


o o i •• 













- 






1 







«o 


a n-\\ 




^1 



- '■ (A ti-K) = 



1 o 
o 1 



•A 



-i. 



As can be seen, the obtained matrix has still the controllability canonical form. 
The coefficients of the closed-loop characteristic polynomial are thus obtained 
without calculation, since in that form the last row of the system matrix, here 
A CL , is composed of the characteristic equation coefficients, except the one of the 
highest power term, with change of sign, according to Rule 1 of Sect. A. 7. 2 of 
Appendix A. Thus the following theorem: 

Theorem 1.1. Suppose that, in the control system of Fig. 1.3, the plant is con- 
tinuous-time, single-input, and is given in controllability canonical form, with the 
characteristic polynomial 



and that, for the closed-loop system, the characteristic polynomial 

n , n— 1 i i i 

s +P n -iS + ■■■ +PiS + p 
is prescribed. Then, the L matrix which will implement it is given by 
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L = £ c =(p -a Pl~<*\ 



Pn-l-Ctn-l) ■ 



13 
(1.23) 



1.2.2.2 Plants in Arbitrary State-representation (Ackermann's 
Formula) 

If the plant is controllable, it is always possible to transform it to the controllabil- 
ity canonical form by means of the following coordinate transformation (see 
Sect. A.7.2 of Appendix A): 

z = Vx, 

where z is the state vector characterizing this canonical form, and where the in- 
verse transform matrix, V = T _1 , is given by the following formula (equation 
(A.32) of Appendix A): 



C T 

q c 

qjA 



iq'A"- 1 



where qj is the last row of the controllability matrix inverse, thus of Q c . 

In the controllability canonical form, the control law is written, according to 

(1.21), as 



which yields 



U — -c /-> Z , 



u=-£ l c Yx. 



With the use of (1.23), the feedback matrix £ c has then, in an arbitrary state rep- 
resentation, the following expression: 



,V = ( Po -a p x -a x ••• p n _ x -a n _ x _ 



T 

q c 

qjA 



[1c A ) 



- (po - «o) qJ + (pi - a \ ) qJ A - 



•(A-i-fln-OqjA- 1 . (1.24) 
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To continue this calculation, a well known theorem from matrix analysis will be 
used. 

Cayley-Hamilton Theorem. Every square matrix A, of size (nxn), satisfies its 
own characteristic equation. In other words, if the characteristic equation of A is 

p{\) = det(AI - A) = A" + a^X"' 1 + ■■■ +a 1 X + a =0, 

then the following holds: 

p(A) = A"+a n _ l A n - 1 + ••• + a 1 A + a o I = . (1.25) 

It is known moreover that two similar matrices have the same eigenvalues. This 
is thus the case of A and A c = V AV (see Sect. A. 7.1 and Equation (A.31) of 

Appendix A). They will thus have also the same characteristic polynomial. Ac- 
cording to the Cayley-Hamilton theorem, A must thus satisfy the characteristic 
equation of A c , 

A"+a n _ 1 A"~ + ■■■ +q, A + a I = 0, 

yielding 

— a \ — fljA a n _ l A"~ =A", 

and, by left multiplication by qj , 

-a qj - a, qjA a n _ x q T c A"" 1 = qj A" . (1 .26) 

Substituting (1.26) in (1.24) yields then the remarkable following result [Ack72]: 

Theorem 1.2. If the plant x = Ax + bw of a single-input control system is con- 
trollable, and if the closed-loop characteristic polynomial 

p(s) = s n +p n ^s"- l + ••• + Pl s + Po 

is prescribed, one must choose 



£ J =p q T c +p l q T c A+ ••• +p n _ l q T c A n - l +q J c A n 



„T 



(1.27) 



= qc^(A) 
where q c is the last row of the controllability matrix inverse, Q^ , and is thus 
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determined, according to (A. 33) of Appendix A, by 
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"Mo 



q c =^u ••• l)(b Ab ••• A" _1 b) \ 
This is the Ackermann 's theorem, and (1 .27) is the Ackermann 's formula. 



1.2.2.3 Example 

Consider the single input, single output system of third order of Fig. 1.4. Let us 
design the state-feedback control of this system, i.e. calculate the constants £ l , 
£ 2 and £ 3 , which are the elements of its (1x3) feedback matrix 
£ T = (£j £ 2 £ 3 ) , and determine thus its control law u = —£ T x . The following 
poles (eigenvalues) should be imposed to the closed-loop: s l2 =—2±j2, 
s 3 = — 10 . Both methods described above will be used. 




Fig. 1.4 State-feedback control of a third order system. 

Direct Calculation: The system order being here sufficiently small (n = 3) , the 

system of equations (1.19) can be solved directly. The state-variable representa- 
tion of this system by means of the state variables indicated in the figure is ob- 
tained by direct inspection: 



x = Ax + b u 



y = c x 



with 



A = 



'0 


1 


0' 




'°1 





-1 


1 


, b = 











-2 

J 




1 



c=l 



Let us calculate the characteristic polynomial of the closed-loop system. Since 
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b£ J 



0' 






'0 





o N 





(<1 


e 2 e 3 ) = 











\ J 






A 


^2 


*3, 



the closed-loop system matrix is given by 



A-br 



( 1 
-1 





1 

-2-£ 



3 J 



and the corresponding characteristic polynomial has the following expression: 



det[sI-(A-b£ T )| 



s -1 

s + l -1 

^ e 2 s +2+e : 



s 3 + (£ 3 + 3> 2 +(£ 2 +£ 3 +2> + £ 1 . 



By coefficient identification of this polynomial with those of the prescribed one, 
namely 

p(s) = (s + 2 + j2)(s + 2- j2)(s + 10) = (s 2 + 4s + 8)(s + 10) 

= s 3 + Us 2 + 48s + 80 = s 3 + p 2 s 2 + p x s + p , 

one obtains readily: 

p 2 =l4 = £ 3 + 3, 

Pl =4& = e 2 +e 3 +2, 
Po = m=i l , 

which yields, by solving this system of linear equations in the unknowns l x , £ 2 
and £ 3 : 



£ 1= 80, £ 2 =48-11-2 = 35, € 3 =14 — 3 = 11 



Finally: 



u = — 80x, — 35x 2 — 1 lx 3 . 
Use of Ackermann's formula: Calculate successively 



1 .2 Pole Placement Method 



17 



Ab = 



'0 


1 


% 







' o s 





-1 


1 





= 


1 








-2^ 


1 




r 2 i 



and A 2 b = A Ab 



'0 


1 


o N 


' 




' r 





-1 


1 


1 


= 


-3 








-2^ 


-2 




4 



and determine then the row vector qj = (^ </ 2 g 3 ) according to (A. 33): 



qjb = 
qjAb = 
qjA 2 b = 1 



% =0, 

g 2 — 2^3 = => q 2 = , 

(ft — 3^2+4^3=1 =>■ ^=1. 



Therefore: 



q c = (i o o) 

q?A = (l 0) 



qcA 2 =qjA-A = (0 1 



qjA 3 -qjA 2 -A = (0 -1 1 



From (1.27), 



1 0] = (0 1 0) 
-1 1 
0-2 

f0 1 0) 
-1 1 
0-2 

f0 1 0) 
-1 1 
0-2 



^ = Poll + Me A + A>q I a 2 + qJ a 3 

= (80 48-14 + 1 14-3) = (80 35 11). 



= -1 1 



= 1 -3 



x/? = 80x(l 0) 
x/7 1= 48x(0 1 0) 



x/? 2 =14x(0 -1 1) 



xl= (0 1-3) 



This result is identical to the previous one. 



1.2.2.4 Case of the Discrete-time Systems 

The pole placement method described above applies also to discrete-time systems. 
As a matter of fact, it is for this type of systems that it has been established ini- 
tially [Ack72]. We will therefore here only state this theorem. The choice of the 
closed-loop eigenvalues can result from a simple transposition of the values \ Li 
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chosen for a continuous-time design, by the relation v Li = e Xu Ts , where T s is the 
period at which the continuous system has been sampled ([AsWi97], [GoOs03]). 

Theorem 1.3. If the single- input plant x k+l =Ox t + yu k is controllable, and if 

it is desired that the state-feedback control law u k = —£ x k yields the closed-loop 
system characteristic polynomial 

p(z) = z"+p n _ 1 z"~ l + ■■■ + p 1 z + p =(z-v u )---(z-v Ln ), 
one must choose 



£ 1 =P q T c +P l q T c O+ ••• +/V 1 q>"- 1 +qJO" 



= qc/>(*) 



(1.28) 



where q c is the last row of the controllability matrix inverse, Q c , and is thus 
determined by: qe=(0 ■•• l) (y Oy ••• 0" _1 y) . 



1.2.3 Multiple-input Systems 

The problem of determining L remains solvable if the plant is controllable, but, as 
seen in Sect. 1.2.1, the solution is no longer unique. This can be put to advantage 
to impose additional conditions to the closed-loop system. 

In the methods which will be presented in Sections 1.4 through 1.7, in addition 
to the wished eigenvalues a particular structure will be imposed to the closed loop. 



1.2.4 Plant Zeros and Closed-loop Zeros 

The previous theorems for SISO systems, as well as those which will be presented 
later concerning MIMO systems, allow shifting the poles (actually, the eigenval- 
ues) of a system to any point of the complex plane by means of a constant, suita- 
bly chosen, state feedback, if this system is controllable. 

The question which arises then is the following: what happens to the plant ze- 
ros during this pole-shifting process? That question is of utmost importance, since, 
if the zeros move arbitrarily, they can deteriorate the dynamic behavior, which 
was aimed at by choosing a new pole configuration. An extreme case, e.g., would 
be that of a zero falling accidentally at the same place as one of the shifted poles. 
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Theorem 1.4. The open-loop zeros are not modified by the pole placement pro- 
cedure. In other words, the zeros of a system, which is closed by a state-feedback 
matrix L, are the zeros of the initial plant. 

Proof. The zeros of a continuous-time system, for which it is assumed that p = q 
and D = , and which is closed by a state feedback, are the ^-values solving the 
following equation, see Appendix A, (A.29): 



sI-A+BL B 
C 



= 0. 



The value of this (n + p) X (n + q) determinant remains unchanged if one adds to 
the n first columns the last column, right multiplied by the L matrix: 



sI-A B 

C 



According to (A.29) again, this equation yields the zeros of the initial plant. There 
is thus identity between these zeros and those of the closed-loop system. 



1.3 Finite Settling-time Design for Discrete Systems 



1.3.1 Problem Description 



This design method, also called dead-beat control, applies only to discrete -time 
systems. Consider such a system with a scalar input, 



H+l 



-Ox k +yu k , 



where O is of size (nxn). 

The goal is to find a control sequence, or control law, [u ,u x , ■••,u k ] , which 
transfers the system from an arbitrary initial state x into the final state x ,■ = 
(the origin) in the minimum number of discrete time steps. 
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1.3.2 Solution: Algorithm ofMullin andDe Barbeyrac 

This problem can be solved in n discrete time steps at most, provided the system is 
controllable and the state-feedback control is possible [MuBa64]. 

Proof. Let u k = — £ x k be the searched control law. In order to establish it, let us 
calculate the state at successive discrete times: 

x (arbitrary initial state) 



x i =Ox +yu =Ox -y£ T x =(O-y£ T )x 

x 2 = ®x l + yu x =Oxj -y£ T x l = (<&-y£ T )x l = (O - y i 1 f x 
= <^(<^-y£ J )x -y£ J (<^-y£ J )x 



<& -(y Oy" 



(eT 



T\\ 



t(0-yt) 

T 



x 3 = Ox 2 +yw 2 = <Dx 2 -y£ J x 2 = (0-y^ T )x 2 = (O - y £ J ) 3 x 



[o[o 2 -(y oy)^ t (o-y^ t ))1-y^ 1 (3 > -y^ 1 ) / 



O Oy O y 



(eT 



T \\ 



t(0-yt) 



-y£ T (<P-y£ T ) 2 



O — y Oy O y 



(„T 



T\2"l 



t(0-y£ ') 
^ t (0-y^ t ) 



v Q , ... 



and so on. We can extrapolate easily this result to step n: 



O" — (y Oy ••• 0""'y 



T\n-H 



t{<S>-yt) 



£ l (d>-y£ l ) 



The state at this step will be equal to the state space origin (the final state), 
x n = , and this V x , if and only if 
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O" — y Oy ••• O y 



^T 



Tn«-1 



£ l (0-yt) 



£ T (0-y£ T ) 



0, 



or: 



( £ J (0-y£ J ) n - l] 



£ J (0-y£ T ) 



y Oy ••• 0" _1 y) <D" 



The matrix to be inverted in the second member of this equation is nothing else 
but the controllability matrix Q c of the system (see (A. 13), Appendix A). Its in- 
verse exists therefore if the system is controllable, hypothesis which has been in- 
troduced at the beginning of this section. The final result is thus the following: 

The desired control law £ T is the last row of the matrix 



Y Oy 



0" _1 y 



<D" 



i.e., with qj being defined as the last row of Q c l (see Sect. A.7.2): 



qJO" 



(1.29) 



Remark 1.5. In the general case, a sequence of n steps is thus necessary for this 
control law to bring the system to its final state. However, depending on the initial 
state, it may happen that less than n steps are sufficient. 

Remark 1.6. The remarkable result obtained derives also from Ackermann's theo- 
rem, as a special case, by choosing to place all the discrete system poles at the ori- 
gin of the z-plane ( // = 0, V i = 1, . . . , n ), i.e. by prescribing the characteristic 
polynomial 



P(z) 



What we have obtained here by a different derivation is the same as the special fi- 
nite settling time variant of a general RST structure [GoOs03]. 

Remark 1.7. This type of control law, due to the fact that it drives the control sys- 
tem to equilibrium in a minimum number of discrete time steps, provides fre- 
quently control signals u k of high level, at least during the first steps. For this rea- 
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son it has acquired, according to [AsWi97], "an undeservedly bad reputation" . In 
the case of the digital control of a real continuous-time plant, it is indeed often 
possible to apply this method with some sacrifice, e.g. by increasing the sampling 
time T s to such a value that the first control signal does no longer exceed the al- 
lowed limit, if the choice of T s is not dictated by other considerations. Another 
possibility, if the sampling time cannot be adjusted, consists in imposing a con- 
straint to this first control signal, which obliges then to give up the control in a 
minimum number of steps, and to add one or several sampling steps beyond n. 



1.4 Simple (or Reduced) Modal Control 

1.4.1 Definition 

By the similitude transformation 

x = Tx* =x* \ { + x* 2 v 2 + ••• +x*v„, 

where V[ , v 2 , ■ • • , v„ are the eigenvectors of A, it is possible to diagonalize the 
system matrix A if all its eigenvalues are distinct (see Sect. A. 7. 4, Appendix A). 
In the new basis, the basis of the eigenvectors, the state differential equations 

x * =\ x i + u i 

are then entirely decoupled. If the plant has p = n inputs and if the control inputs 
u* are independent from each other, each state variable x* , and consequently 

each mode x* \ t , can be controlled separately. This is why this control is referred 

to as modal control. 

In practice, it will generally only be possible to come close to an ideal modal 
control, first because there may exist multiple eigenvalues and the state variables 
associated with them are not completely decoupled, second because usually the 
plant does not have as many control inputs as state variables. 

1.4.2 Simple Modal Control Design Method 

In the remainder of this section, the eigenvalues will be assumed all distinct. Since 
p control inputs are available, we will try to control individually p modal coordi- 
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nates. Due to the formal similarity of the expressions, only the continuous case 
will be considered here. 

The plant state representation, in its most general form 

x = Ax + Bu, 

where size(A) — nxn and size(u) = p X 1 with p < n , is transformed into its di- 
agonal form 



x* = Ax* +Bu 



(1.30) 



by the transformation 



with 



x = Tx* 



A = T _1 AT 



and 



B = T _1 B 



( Tl 
W 



B 



w^B 



wlB 



(1.31) 



where T is the modal matrix and where w ; denotes the ;-th row of its inverse 
T _1 (see (A.30) and Sect. A.7.4 of Appendix A). 



1.4.2.1 Choice of the/; Modal Coordinates Fed Back to the Input 



This choice depends on the real data of the plant to control. It is common to use 
for the feedback control those coordinates which have associated eigenvalues 
closest to the imaginary axis, since they are the ones which determine principally 
the dynamic behavior of the plant and must therefore be controlled in priority. 

In cases where essentially the plant behavior with respect to a disturbance is of 
interest, the modal coordinates which are most influenced by this disturbance 
should be retained. 



Let us group them together in a vector x* = I xj* • • • x* J , and call \*„_ p the 
vector of remaining modal coordinates. (1.30) can then be written as 
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n—p 



P 

A 



o s 


X 




f * \ 

B 




P 




p 


n—p 


X n-p 


+ 


V n ~P , 
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u, (1.32) 



where: A, 



B 



B 



This yields 



= (pxp) north-west submatrix of A; 

= (n — p)x(n — p) south-east submatrix of A; 

= submatrix of B made of its first/? rows, thus of size (px p); 

= submatrix of B made of its last (n — p) rows. 



K = A p x p+~ R p u 

X n-p = A-n-p X n-p '"n-p U 



(1.33) 



The first of these two equations can be written 



by introducing 



i P =A P xr P +u ' 



u^B^u 



(1.34) 



(1.35) 



1.4.2.2 Principle of the Simple Modal Control 

We want now to determine the feedback of these/? first modal coordinates in such 
a way that, simultaneously, 

1. the given eigenvalues Aj,---,A are replaced by the desired eigenvalues: 

2. the closed-loop system remains diagonal as far as these p state variables are 
concerned; this is the principle of modal control: each mode is controlled inde- 
pendently. 

The state equation of the closed-loop system, in regulation behavior (y ; . = 0) , 
must therefore have the following form for the/? involved coordinates: 



K = a l p X * P > where A Lp = 



A, 







••• A 



Lp) 
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and, consequently, the following equality must hold: 
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A p x* p + u* = A Lp x* p , 



yielding 



-( A p- A L P )X* P 



^1 ~^L1 



\ ~^Lp 



V X P 



(1.36) 



Note that u* is a fictitious control vector, which does not exist in the real sys- 
tem. It depends on u according to (1.35). This equation must be solvable in u, 
which is the real control vector acting on the plant: 



b;'u 



(1.37) 



For that purpose, B p , which is a square (p x p) matrix, must be regular. Now 
B is a submatrix made from the p first rows of B , which is itself an (n x p) 

matrix of rank/?, since B has this rank and since T~ is a regular matrix (thus 
composed of linearly independent rows w ; T ). It is therefore possible to find in B 
a number p of linearly independent rows. If these rows are not the first ones, it is 
always possible to renumber the state variables for this to be the case. Of course, if 
a set of p modal coordinates to be controlled has already been chosen, it may then 
be necessary to modify this choice and, eventually, to drop one or another of these 
variables for feedback control. 
Equations (1.36) and (1.37) yield 



B p (A p -A Lp )x* p , 



(1.38) 



then, by substitution into the differential equations (1.33), 



A x* 



x n-p = "n-p"p \ A p ~ A Lp) X p + A n-p X n-p 



(1.39) 



The first equation (1.39) shows that the p first modal coordinates are indeed 
controlled individually. The corresponding eigenvalues, X Ll to X Lp , can be im- 
posed arbitrarily. 
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1.4.2.3 Influence on the Other Modal Coordinates 

The second equation (1.39) shows that the (« — p) remaining modal coordinates 

are influenced by the p first ones. It is therefore no longer possible to control them 
arbitrarily. 

What can we say then about the closed-loop stability? The closed-loop eigen- 
values are obtained by solving 



sI p- A Lp 







*n-p*pH A p-^Lp) Sl n _ p -A n _ p 



= 0. 



thus 



i.e., explicitly 



k I P - a l p x \s K-p - K-p = ° > 



(s-X n )-(s-X Lp )(s-X p+l )-(s-X n ) = 0. 



This means that the remaining (n — p) closed-loop eigenvalues are equal to the 
plant eigenvalues X p+l ,■■■ ,X n , thus have remained unchanged, whereas the p first 

ones have been shifted from A ; to X Li by the simple modal feedback. 

Conclusion: the closed-loop stability is not endangered by the repercussion of the 
control of the p first modal coordinates on the remaining (n — p) ones. 



1.4.2.4 Simple Modal Controller 

The entire previous development has taken place in the state space spanned by the 
eigenvectors. We must now go back to the initial state representation. With 



r'x 



,j\ 



w„ 



we get from (1.38): 



-B p - l (A p -A Lp ) 



\= Lx 



w 



p ) 
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The simple modal controller is thus given by the (p x ri) matrix 
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w^B 



w T B 



^1 ~^L1 







X p-\ P ) 



i'J 



where the employed notations have the following recalled significations: 



w 



= 2-th row of the inverse transformation matrix, T l , 
| A, | = p first plant eigenvalues, 

{A i( }.._ = p first closed-loop eigenvalues (prescribed values). 

Remark 1.8. The present control is again a proportional control law. 



(1.40) 



1.4.2.5 Block Diagram 

The previous modal control applied to the state vector is represented in Fig. 1.5. 

1- 

u , *U x 

9 — 



x = Ax + Bu 



^^ 



HsT H A '- A ») ^- ; T 



Modal Controller 
Fig. 1.5 State feedback by modal control. 

The drawback of this diagram is that it does not show the decoupling of the/? first 
modal coordinates. Let us thus transpose the previous equations to the s-plane. 



1.4.2.6 Application of the Laplace Transform 

Applying the Laplace transform to the two equations (1.39), 
sX* p (s)-x* p (Q) = A Lp X* p (s) 

sK- P (*) - K- P (o) = -K-p b; 1 (a p - a Lp ) x; ( S ) + a„_ p x:_ p (*) 
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where X* p (s) = C\x* p (t)\ and X* n _ p (s) = C\\* n _ p (t)\, we obtain: 



X*(s) = (sI-A LD r 1 x*(0) 



LpJ 



Xl_ p (s) = (sI-A„_ p y l Wl_ p (0)-B n _ p B p -\A p -A Lp )X* p (s)\ 



Since the matrices to be inverted are diagonal, so are their inverses. They have 
the diagonal elements 



for i = 1, ... , p and for i = p + 1, . . . , n . 

-A,, s-A, 



Taking into account the fact that 



x*(0) = T _1 x(0): 



(„J) 



w„ 



x(0) f 



the block diagram of Fig. 1.6 is obtained. 



x(0) 



x,*(0) 



r \ 



s-A, 



x;(s) 



4(°), 



s — X. 



■Lp 



( \ 

T 

w p+1 



<-M 



Modal 

Analysis 

(T" 1 ) 




X' p (s) 



K + ^ 



l^i -&n | \^p~^Lp\ 



B » P B ; 






1 




- s "Vi 








1 




.v-/l„ 



X(.v) 



*:w 



Y 

Control in 
Modal Coordinates 



Modal 

Synthesis 

(T) 



Fig. 1.6 Diagram of the modal control detailed as to the modal coordinates. 
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This figure shows clearly that: 

• the first p modal coordinates are controlled individually and have acquired 
thereby the eigenvalues \ Ll to X Lp ; 

• the (« — p) remaining modal coordinates are influenced by the first p ones, but 
even so, they have not lost their eigenvalues \ p+l , ■ ■ ■ , X n . 

1.4.2.7 Special Case: p = n 

It is then possible to avoid the modal transformation T if one wants only to shift 
the eigenvalues. By writing, indeed, 



one obtains directly 



i = (A — BL) x = A L x , 



L = B" 1 (A-A i ), (1.41) 



since in this case B is invertible (rank(B) = p = n) . 



1.5 Input - Output Decoupling Method 



1.5.1 Initial Hypotheses 

The approach presented here, which has been introduced by [FaWo67], aims at 
producing a closed-loop system in which each input influences only one output, 
therefore its denomination of input - output decoupling control. It applies thus 
only to systems having a number of inputs identical to the number of outputs: 
p = q. 

That restriction is perfectly justified: for the case where p<q , the decoupling 
would anyway not be possible, and, for that where p> q , it is easy to revert to 
the case p = q by dropping (p — q) plant inputs for control purposes. It is as- 
sumed furthermore that rank(B) = p and that rank(C) = g, i.e. that the plant 

state model has neither redundant inputs nor redundant outputs, as it should be the 
case in every state representation (see Equations (A. 7) and (A. 8) of Appendix A 
and the associated conditions about the ranks of B and C). Finally, as will be 
shown in the sequel, the plant must be not only controllable, but also observable. 
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1.5.2 Order Differences of a System 

The following developments concern a MIMO continuous-time plant, described 
by a state representation of type (A. 7) of Appendix A, with D = . They are quite 
similar in the discrete-time case by simple substitution of z to s. The goal is thus to 
obtain a closed-loop transfer matrix which has the following shape: 



G a (s) = 



G,( 5 ) - 
... GJs) 



<-r 



(1.42) 



where the q SISO individual transfer functions G t (s) have the form 

G ( S)= NM (1.43) 

AW 

with 

NAs) = k t = constant , (1-44) 



and 



£>. (s) = s s > + g iA -iS Si '+... + q ul s + q ifs = (s - p, x )...(«- p iA ) , (1-45), 



where p iJ (j = l,...,6 i ) are the desired poles for G t (s) and constitute, together 
with all the other poles for i = \,...,q, the set of poles (eigenvalues) imposed to 

the decoupled closed-loop system which will result from this design. 

The restriction (1.44) imposed to the numerator of G t (s) is easy to understand. 
It was proved in Sect. 1.2.4, Theorem 1.4, that the plant zeros remain unchanged 
when its poles (eigenvalues) are shifted by a state feedback. The zeros of G CL (s) 
are thus the same as those of the open-loop transfer matrix 
G 0L (s) = C(sI — A)~ l Ti . Therefore, if one of the transfer functions G t (s) in 
(1.42) had a zero s = rj , all the elements of the 2-th row of G 0L (s) should vanish 

for that same value of s, which is extremely rare in practice. A second justification 
of this restriction will be given a posteriori. 

The number 6 t of poles contained in G t {s) is, as to itself, dictated by the fol- 
lowing consideration. 

Definition 1.1. The order difference of the output y i is defined as the lowest or- 
der S t of the derivatives of y t (t) , thus of the first one encountered in order of 

successive derivatives y t ' (t) , on which the input vector u, thus the reference 
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vector y r in the case of a decoupled control, exerts a direct action, its action on 
all the derivatives of lower order occurring only through the state vector x. 
This value is the smallest integer number S t for which holds 



cjB = 



c'AB = 



c, T A*'- 2 B = 

Ja* _1 b*o 



(1.46) 



where c^ is the 2-th row of the output matrix C, thus yielding y i = cjx . 

The order difference of the MIMO system with state representation A, B, C, is the 
sum of the order differences of its various outputs: 

<5 = <5, + ... + <5 ? . 

The proof that (1.46) yields the order difference <5 ; of the output y t (f) consists 
in differentiating this output successively and applying at each step the corre- 
sponding line of (1.46): 



h = c, T x = c, T Ax + c^Bu = c^Ax, 

y\ = cjAx = cjA 2 x + c^ABu = c, T A 2 x 



(1.47) 



^-D =c t a ^-i Xj 



Jil-J^i 



c, 1 A < *x + c l 1 A < *- 1 Bu 



Ta4-1i 



y] ' is thus well the first derivative on which u acts directly. By combining it 
with those coming from all the other outputs, with their respective order differ- 



ences for i = \,...,q, in a fictitious output vector y =[y\ " ■•• y, 
can write y = C x + D u , with 



C = 



'c*A* ' 


( 




, D* = 


c T A* ? 





cfA^B 



c'A « B 



(1.48) 
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1.5.3 Decoupled Control According to Falb-Wolovich 

The substitution of the general expression (1.2) of a state-feedback plus gain- 
compensation control, namely u = — Lx + My r , into the last equation of (1.47) 
yields, for i = \,...,p since, as mentioned above, p = q: 



}'i 



( < %)-Cp t A' 5 « _, T A*n 



(c, 1 A"' - c, 1 A^'BI^x + c/ A° r BMy r 



Ta«,-1i 



(1.49) 



With the purpose to obtain that the output y i be influenced only by its attrib- 
uted reference y r i , it is then logic to let 



, T A S /-! 



c/A^'BMy^.v .=(0 - k, - 0) y r 



(1.50) 



where k i is an arbitrary constant, and this for i = \,...,p . In matrix form, this can 
also be written 



'c^-'B' 


i 




My r = 


c^A^-'B 





\ o ••• o s 

o ■•. ; 

k pl 



Yr- 



Since this equality must hold whatever y r , it leads with the use of (1.48) to 



D*M = diag(A: 1 ,...,^). 



(1.51) 



In order to complete the decoupling of (1.49) it is further necessary to express the 
coefficient of x in this equation as a function of y t and its derivatives. One degree 
of freedom remains available to do this, the L matrix. It is enough then, to let 



~T l Si T l ft—1 



(Si-l) 



(C/A -' -^A ^BL)x = -(q iA _ l y}°'- l >+... + q^y i +q ifi y i ) , (1.52) 

and to replace the successive derivatives of y t by their values derived from (1.47), 
to see that L must then satisfy 

(c, T A^ - c^A^- 1 BL)x = -{q iA _ x cjA^x + ... + <?,, c^Ax + q lfi c]x) , 
thus also, since this equality must hold for any x, 
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i,« r l l i 



T A 4-1 



c A 



■4li c J a +<1lq c I 



= c/Z).(A), » = !,...,/>, 



where the second equal sign results from (1.45). Regrouping these p equations in 
matrix form with the use of (1.48), we obtain 



D L 



CiA(A) 



(1.53) 



1#>„(A)J 
Substituting now (1.52) and (1.50) into (1.49), we find 

(Si) i (&— 1) i i • i ? 

y) + % Si-iy} +■■■ + %\ y t + q ifi y, = k y r> ,■ , 

equation which confirms that the decoupling is total, and, at the same time, re- 
trieves, by applying the Laplace transform, the expression of the transfer function 
G i (s) = Y i (s)/Y ri (s) given by (1.43) to (1.45), for i = \,...,p . 

The expected control law will thus be obtained by solving (1.51) and (1.53) for 
M and L, which will be possible only if D is invertible. 
Let us summarize these results: 

Theorem 1.5. Consider a plant \ T . It is possible to design a 

[j,=c,.x, i = \,...,p 

control law u = — Lx + My r that decouples the closed-loop system if, and only if, 



detD =det 



JA^B 



^A"' B 



= 0, 



(1.54) 



where 6 t is the order difference of the output y t , determined by (1.46). 
The parameters of this control law are then given by 



L = D 



c7a(a) 



[*pD p (A) 



D 



^J2ii, v A v 

^E^ A " 



</=() 



M = D 



h ••• 



••• k„ 



(1.55) 
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The closed-loop system is then equivalent to the following p distinct SISO systems: 

Y,(s) = - J Y ri (s), i = l,...,p. 

s' +q iA -iS' +... + q itl s + q ifi 

The still free parameters q iv can then be determined separately for each of these 
systems, e.g. by pole placement, and one can impose additionally k t = q t in or- 
der to guarantee the static accuracy of each of them. 



1.5.4 Control Implementation 



1.5.4.1 Case where 8-n: Maximal Order-difference System 

Since we assign in this case 8 = <5, + . . . + 8 = n poles to the various branches of 

the decoupled system, poles which are at the same time eigenvalues of the latter, it 
is thus possible here to shift all its eigenvalues. An example will illustrate this 
case in the solved exercises of this chapter. 



1.5.4.2 Case where 8<n 

The whole set of SISO systems equivalent to the closed-loop system has here a 
number 8 of eigenvalues, lower than that of the closed-loop. It has been shown, 
[F6190], [Rop90], that the control law determined by this method assigns then to 
the closed-loop system zeros in its transfer matrix, which cancel these missing ei- 
genvalues, so that they do not appear any more in the decoupled structure. 
[Rop90] has even shown that one feasibility criterion of the decoupling by state 
feedback, equivalent to (1.54), consists in verifying whether the plant possesses 
exactly n — 8 zeros. For a discussion about internal stability, see Sect. 1.6.5, Re- 
mark 1.15. 



1.6 Complete Modal Control 

The method described in Sect. 1 .4 allows shifting only a number p of eigenvalues 
equal to that of the plant inputs, very often lower than the plant order n in the case 
of MIMO systems. 

The method which will be described hereafter, and which has been introduced 
by [Rop82], allows assigning all the closed-loop eigenvalues, therefore its name 
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complete modal control, also parametric controller design. Unlike the simple mo- 
dal control, the modal quantities of the closed-loop system, not of the plant, will 
be used here. 



1.6.1 Control Law Derivation (MIMO Case): 
Roppenecker's Formula 

Let us consider again a linear time-invariant system, x = Ax + Bu in the continu- 
ous case, or x t+1 = Ox k + Tu k in the discrete case. Assume in the following that 

1. the eigenvalues X i ,...,X n ofAorof O are all distinct; 

2. a control law u = Lx has been synthesized, which yields the closed-loop ei- 
genvalues X L1 ,...,X Ln , also all distinct. The way L was obtained, e.g. by pole 
assignment or even by LQ design as described in Chap. 3, is unimportant. 

The case of multiple open-loop eigenvalues has been taken into account in 
[Rop86], but involves deeper calculations than the ones to be presented here. 

The following will deal only with the continuous case, the calculations being 
easily transposed to the discrete case by merely replacing the A and B matrices by 
O and r , respectively. The eigenvector \ u of the closed-loop system matrix 
A — BL and the corresponding eigenvalue X Li satisfy the equation 

[A L! .I-(A-BL)]v L ,=0, i = l,...,n, (1.56) 

which can be written also 

(A-A a I)v a =BLv a> / = 1,...,«. (1.57) 

Let us introduce n parameter vectors, of size p, defined by 

p, =Lv Li , / = 1,...,« . (1.58) 

(1.57) becomes then, successively 

(A-A L! .I)v L ,=Bp„ i = \,...,n, (1.59) 

v i ,=(A-A i! .I)- 1 Bp,, / = !,...,«. (1.60) 
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The existence of (A — A I; I)~ is guaranteed if we assume temporarily that the 

X Li haven been chosen different from the A, , thus that all A eigenvalues have 

been shifted. This restriction will be removed in Remark 1.12 below. 
Let us gather the p, and v Lj vectors in two matrices: 

( Pl -. p, I ) = L(v il - y Ln ). (1.61) 

Since the \ Li are all different, the eigenvectors \ Lj are linearly independent. 

Let us go now the reverse way, by choosing arbitrarily the \ Li 's and the p, 's, 

L remaining to be determined. The two previous equations lead thus to the follow- 
ing theorem: 

Theorem 1.6. Consider a MIMO continuous-time system x = Ax + Bu , assumed 
controllable and having constant matrices A (n x n) and B (nx p ). Let us choose 

arbitrarily n values \ Ll ,...,\ Ln and n vectors Pj,...,p n of size p, with the unique 

constraint that the vectors 

y Ll =(A-X Li iy 1 Bp i , i = \,...,n, (1.62) 

resulting from these two choices are linearly independent. The closed-loop system 
obtained by a state-feedback control u = — Lx with 

l = (pi ••• P„)Ki ••• y Ln )~ l 

-i (1-63) 
= (Pi - PjfCA-A^ir'Bp, - (A-A^ir'Bp,,) 

has the eigenvalues X Ll , ... ,X Ln and the eigenvectors \ Ll ,...,\ " Ln given by (1.62). 

Relation (1.63) is known as the Roppenecker's formula. It calls for several re- 
marks. 

Remark 1.9. If one of the chosen eigenvalues, \ Li , is complex, its complex con- 
jugate \ Li must also be selected. Failing to do so would cause the L matrix to 

contain complex elements. For the same reason, two complex-conjugated parame- 
ter vectors must be chosen in association with this pair of eigenvalues. 

Remark 1.10. The closed-loop eigenvectors \ Li being linearly independent, they 
build a basis of the state space, in which the closed-loop state vector of the free re- 

n 

sponse to an initial state x is expressed by x = ^ x* v Li , with x* = e "' wj ; x 



1 .6 Complete Modal Control 37 

(see (1.66)). Left multiplying this equation by L, we obtain for the opposite con- 
trol vector 



-u = Lx = Y^x*~Lv Li =J2 x i P< ' 
<=i j=i 

according to (1.58). The comparison of these two equations shows that the pa- 
rameter vectors form a basis of the control vectors subspace. Furthermore, the 
control vector u(?) , resulting from the application of the complete modal control 
law to the state vector x(t) at a given time, has in this basis the same components 
as the state vector in the basis of the closed-loop eigenvectors. 

Remark 1.11. Parameter-vector invariance in a regular transformation. 

Let us submit the state vector to a change of basis, defined by a regular trans- 
formation matrix T: x = Tz . The state equations become (Sect. A.7.1 of Appen- 
dix A) 

z = Az + Bu, A = T _1 AT, B = T _1 B, 

and the control law writes 

u= Lx= Lz, with L = LT. (1.64) 

The eigenvalues of a matrix remaining unchanged in a change of basis, the 
closed-loop eigenvectors satisfy, in the new coordinates, the following equation: 

[A L ,.I-(A-BL)]v Li .=0, 

which can also be written 

[A L ,.T _1 T-(T _1 AT-T _1 BLT)jv t ,. =0, 

thus, by left multiplication with T, 

[A L ,.-(A-BL)]Tv L ,.=0. 

The comparison of the last equation with (1.56) shows that 

v^T-'v,,.. (1.65) 
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The eigenvectors are transformed thus the same way as the state vector coordi- 
nates. For the parameter vectors in the new coordinates, however, their definition 
(1.58) combined with (1.65) leads to 

p,=Lv ii= LTT - l y u =-Ly u =p i . 

In conclusion, the parameter vectors are invariant in a regular transformation, 
as are the eigenvalues, whereas the eigenvectors and the state-feedback matrix are 
not (see (1.65) and (1.64)). They reflect thus, in the same way as the eigenvalues, 
an intrinsic property of the control system, and are also called for this reason in- 
variant parameter vectors [Rop81], [Rop90]. 

Remark 1.12. While writing (1.60), we have assumed that all initial eigenvalues 
of the plant were shifted, which guaranteed that det(A — \ Li I) * 0, Vi = 1,...,« . 

This restriction can be removed easily. 

Suppose, indeed, that we would have wanted to keep a given initial eigenvalue 
A ; . It is then logic to associate with the choice \ Li = A, the requirement that the 

eigenvector corresponding to this eigenvalue remains itself also unchanged: 
\ u = v ; . Then, according to (1.59), 

(A-A^v^Bp,., 

thus 

= B Pl , 

since v, is the eigenvector associated with A, . As a result, p ; = , since B is of 
full rank. This leads to the 

Complementary Rule to Theorem 1.6. If an eigenvalue of the initial plant 
should not be shifted (X Li = A ; ) , let p ; = in the formula (1.63) which yields L, 

and replace there 

v ii .=(A-A i! .I)- 1 Bp,. 

by the eigenvector v ; of the plant. 

Remark 1.13: choice of the parameter vectors. In the design of a state feedback 
for a MIMO system, the degrees of freedom, which are left once the closed-loop 
eigenvalues have been chosen, are expressed in explicit form by the parameter 
vectors. These vectors are indeed determined, like the eigenvectors, apart from 
one multiplicative factor. They represent thus a total of n(p — 1) parameters, 

which, added to the n prescribed eigenvalues, make up for the total number np of 
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design parameters required to determine the np elements of the L matrix in the 
MIMO case. It is obvious that these additional degrees of freedom do not exist in 
the SISO case. 

As mentioned previously, the parameter vectors are chosen arbitrarily, and this 
choice can be completely at random. This way of doing yields the highest prob- 
ability that the eigenvectors resulting from (1.62) be, as required, linearly inde- 
pendent. 

It is however possible to refine this choice, in order to fulfill some additional 
requirements for the closed-loop, e.g. specifications of structural nature. One of 
these possibilities will be the subject of the next section. 



1.6.2 Targeted Choice of the Parameter Vectors and the 
Eigenvectors 

1.6.2.1 Influence of the Eigenvectors on the Closed-loop System 
Dynamics 

The dynamic response of the closed-loop system state to non-vanishing initial 
conditions x and to an input stimulation, the reference y r (t) , is given, according 
to (1.3) and to (A.18) of Appendix A, where the matrix A is replaced by A — BL 
and u by My r (t) , by the equation 

X( f ) = e ( A " BL )' X + r f e (A-BL)( f -r) B My>) dT 

Jo 
The closed-loop equivalent to (A. 45) can therefore be written as 

*(') = £ Vu^V, x +J2v Li [ t e XLiit - T) yvl i BMy r (T)dT. (1.66) 

■— 1 ' v ' -i v o 

'~ 1 closed-loop eigenmode (— l 

In this equation, \ Lj and w i; represent respectively the eigenvector and the 
left eigenvector (see Appendix A, Sect. A.7.4) associated with the closed-loop ei- 
genvalue X Li . The set of these eigenvalues and eigenvectors constitutes the eigen- 

structure of the closed-loop system. 

By transposing the discussion following (A.45) to the closed-loop situation, it 

must therefore be possible to decouple certain modes e u from certain states, 
thus from certain outputs, by choosing appropriately the associated eigenvectors 
y u [DucOl]. 
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In a symmetric way, it must be possible also to decouple certain transient 
modes e LJ from certain components of the initial state x or from certain com- 
ponents of the reference y r (t), by choosing appropriately the left eigenvectors 
wL of the closed-loop system. The interested reader may find more details about 

this topic in specialized works, such as e.g. [LiKM94], [LKMA94]. We will limit 
ourselves here to the choice of the closed-loop (right) eigenvectors. 



1.6.2.2 Coupled Choice of the Parameter Vectors and the 
Eigenvectors 

Let us start again from (1.59), which can also be written 

(A-A Li I„ -B) Vii =0, i = \,...,n. (1.67) 

If all prescribed values \ Li are different from the eigenvalues of A, the matrix 
(A — \ Li \ n — B j is of rank n and (1.67) represents a system of n linear equations 

in (« + p) unknowns. It is thus possible to choose arbitrarily the value of p of 
them. The resulting system of equations has then one unique solution for the n 
other unknowns. 

The arbitrary choice of p unknowns can be spread indifferently over the com- 
ponents of \ Li and those of p ; . Thus, e.g., the choice of making thej-th compo- 
nent of v Li vanish will cause the mode e L,t to be absent from they'-th component 

of x . A real example developed in the exercises of this chapter will illustrate this 
approach. 



1.6.3 Single-input Systems (p= 1) 

The parameter vectors are here scalars p t • ^ , so as to guarantee the independ- 
ence of the eigenvectors v Li . The matrix in the second factor on the right side of 
(1.63) has then the following expression, with B = b : 



(A-A^irV, ■■■ (A-A iH irV„ 

a ■■■ o' 

(A-A^ir'b ... (A-A^iy-'b) ; \ ; 

I o - Pn ) 
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This formula yields then the state feedback matrix 

{ 1 



41 



L = ^=(A - Pn, 



Pi 



(A-A^ir'b - (A-A^ir'b 



... o 
J_ 

Pn) 



(1 -. l)(A-A LI I)-'b - (A-A^ir'b 



Notice that the parameter vectors have disappeared from the expression of £ , 
which thus depends only on the eigenvalues X Li . This is not a surprise, since in 

the single input case there is no additional degree of freedom. 

The equivalence with the Ackermann's formula is proved by comparison of the 
closed-loop characteristic polynomials in the two approaches [Rop82]. 



1.6.4 Simple Modal Control Contained in Complete 
Modal Control 



As it was seen in Sect. 1.4.2.1, only a number/? of eigenvalues equal to the num- 
ber of the plant inputs are shifted in the approach of simple modal control. With 
the notations of that section, let us choose then the matrix of parameter vectors in 
the form 



P= B-'CA -A LD ) , 



P y-'^-p it -L P > 



(1.68) 



which amounts to taking for the first p parameter vectors 



Pi= s P *i(\-hil i = l,---,P, 



where e,=(0 ••• 1 ■•■ 0) , the non-vanishing coefficient being at the 
j'-th place, and 

p,. = 0, i = p + l,...,n, 

for the (n — p) remaining ones. By substitution of (1.68) into (1.63) the state 
feedback matrix becomes 
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L = V(A p -A L/ ,)(v L1 -. y Ln y\ 

A more in-depth calculation [Rop81], which involves the relation existing be- 
tween the closed-loop eigenvectors, \ Li , and those of the plant, v ; , as well as the 
orthogonality of the left and the right eigenvectors of a matrix, shows then that the 
above expression reduces to 



L = B" 1 (A -A LD ) 



Lp ■ 



T 

w, 



T 
W 

P) 



which is nothing else but (1.40). 



1.6.5 Input - Output Decoupling Control by Complete 
Modal Design 

As in the case of the Falb-Wolovich method exposed in Sect. 1.5, this synthesis is 
possible only if p = q . Starting from these formulae, [Rop90] has shown that, for 

a system controlled by such a control law, if an eigenvector is associated with a 
desired eigenvalue \ Li which appears as one of the poles p jk of the diagonal 

element GAs) according to (1.43)-(1. 45), it satisfies the relation 

Cv Li =ej, if \ u =p Jti , ke{l,...,6 k }, (1-69) 

where e =(0 ■•■ 1 ••• 0) is a unit vector of size q, the only non zero 

element of which is the one at they'-th place. If on the contrary it is associated with 
an eigenvalue which is not equal to any of these poles pj k , it satisfies 

Cv Li .=0, X\ u *p JJc . (1.70) 

Introducing with (1.60) the invariant parameter vector p ; associated with this 
eigenvalue yields 

CXA-A^.ir'Bp^e,, 

thus also 
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where G(s) is the transfer matrix of the open-loop plant. 
In summary, the steps of the synthesis are the following: 

1 . Prescribe for the closed-loop system n — 6 eigenvalues \ Li equal to the n — 6 
zeros of the plant to be decoupled, with corresponding eigenvector \ Li = \ z 
and parameter vector p, = — u z , where x z and u z are the state direction and 
control direction associated with these zeros (Appendix A, Sect. A. 6); 

2. The 6 remaining closed-loop eigenvalues can be chosen arbitrarily, the pa- 
rameter vectors associated with these X Li being given by 

p^-GT^e,. (1.71) 

The choice of these eigenvalues must occur such that a number 6: of them be 
assigned as poles p jk ofthej-th transfer function Gj(s) , with k = l,...,<5 y -; 

3. The matrix L is then calculated according to the procedure of the complete mo- 
dal synthesis (Theorem 1.6, Equation (1.63) ) and the gain compensation (feed- 
forward) matrix M is determined, either by (1.54) and (1.55) or their discrete 
equivalent, or by the general case relation (1.8) or (1.16). 

Remark 1.14. By grouping together equations (1.59) and (1.70), we obtain the 
homogeneous linear equation system 



(A-\ Ll \ B) 
C 



o, 



the solution pairs (A L; ,v Ll ) of which are precisely the plant zeros and their associ- 
ated directions. This confirms the choices which have been imposed in the first 
step of the design procedure. 

Remark 1.15. Case where the plant has "unstable" zeros. 

It would be possible a priori to proceed as previously, in Step 1, by placing on 
these zeros closed-loop eigenvalues in order to cancel them. A complete input- 
output decoupling would indeed result for the closed-loop system, with a diagonal 
transfer matrix, but we would have created this way an unstable internal mode for 
the closed-loop, even though it would be unobservable from the output. 

The complete modal design provides a more realistic solution, by avoiding this 
compensation, even if it means giving up complete decoupling. It is enough then 
to place closed-loop eigenvalues in Step 1 only on "stable" plant zeros, and to join 
the eigenvalues which have become free this way with the other ones, which are 
used in Step 2 of the approach. 



44 



1 State Space Control Design 



Suppose e.g. that such left-aside eigenvalues have been assigned to the y'-th out- 
put. The closed-loop transfer matrix will have the following shape (with p = q): 



G CL (X) = 



[G.is) 



G n (s) 



Gjj(s) 







G J P (°) 



G p (s) 



to compare with (1.42). This approach is called partial input-output decoupling. 

Conclusion: a system can thus be put in a totally decoupled shape, provided it has 
neither any zero with positive real part (or exterior to the unit circle, for discrete 
systems), nor any zero which would be common to all the elements of a same row 
of the plant transfer matrix G 0L (s) — C(sl — A) _1 B , this last situation being 
rarely encountered in practice (see Sect. 1.5.2). 



1.6.6 Disturbance - Output Decoupling by Complete 
Modal Design 

In the previous section, it has been shown that the complete modal design makes it 
possible to realize a decoupling control, which decouples the input-output links. 

It is also possible, in certain circumstances, to eliminate the influence of con- 
stant disturbances on the outputs by means of the complete modal design, thus to 
create a disturbance-output decoupling, [Rop90]. 

Let us consider a continuous-time system with the same number of inputs and 
outputs, p — q, described by 



x = Ax + Bu + Ev 
y = Cx 



where v is a constant disturbance vector of size m, influencing the state through an 
(nxm) matrix E, as illustrated in the diagram of Fig. 1.8, which will bee seen in 

Sect. 1.8.1.4. 

Let us assume that it is equipped with a control law of type (1.2): 
u = — Lx + My r . The closed-loop system is then represented by 



x=(A BL)x + BMy r +Ev 
y = Cx 
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The transfer matrix between the disturbance input v and the output y, thus at van- 
ishing reference input y r , is given by 



G v (s) = C[sI-(A-BL)] ! E. 



(1.72) 



Let us go over to modal representation, with the change of variable x = Tx , T 
being the modal matrix. If the eigenvalues prescribed for the closed-loop system 
are assumed all distinct, its transfer matrix will have in this basis the diagonal 
form A — BL = diag(X L1 ,...,X Ln ), and (1 .72) becomes, by returning to the initial 
representation, 



G» = CT 



S — A, 











A, 



r'E. 



which, by introducing the eigenvectors v Li and the left eigenvectors w i; , the 

transposes of which are the rows of T" 1 (see Appendix A, Sect. A.7.4), yields the 
final formula: 

The desired decoupling will be obtained if one succeeds making that 
G v (s) = , i.e. that Cv Li w{ ; E = 0, V/ = \,...,n , which leads by summation on i 
to the necessary condition 



CE = 0. 

For that condition to be satisfied, it will suffice in fact that 

Cv Ll = for i = l,...,k 
wT,E = for i = k + l,...,n, 



(1.73) 



where the eigenvalues and eigenvectors may have eventually been renumbered. 
Grouping together these two equations yields 



E 



N) 




} n — k 



where N is some (k x m) matrix, from where follows that 



46 1 State Space Control Design 

E = (v L1 - v Lk )N. (1.74) 

This relation shows that the disturbance-output decoupling is possible if, and 
only if, each column of E can be written as a linear combination of the prescribed 
eigenvectors for which C\ Li = holds, in other terms, if, and only if, only unob- 

servable modes are excited by these disturbances. 

Now, according to Remark 1.14, such an eigenvector corresponds to an eigen- 
value which has been placed on a plant zero, and its direction is equal to that of 
the state associated with this zero. 

The design procedure, according to [Rop90], is then the following: 

1. check whether the necessary condition (1.73) for the existence of such a solu- 
tion is fulfilled; 

2. determine then the plant state-vector directions associated with a zero from 
which the columns of the disturbance input matrix E are built, and choose the 
same number of eigenvalues and of parameter vectors, equal respectively to 
these zeros and to the control vector directions associated with these zeros, with 
sign reversal; 

3. the remaining parameters of the complete modal design are chosen according to 
other considerations. 

An application example of this approach will be described in an exercise at the 
end of this chapter. 



1.7 General Formula for MIMO Design 

A more general expression to calculate the state feedback matrix for MIMO sys- 
tems has been established by [BeOs87]. It will be given here only in the continu- 
ous case, since the discrete case is again derived from it trivially by substitution of 
the matrices O and T to the matrices A and B. 



/. 7.1 MIMO Control Law (Becker-Ostertag) 

Theorem 1.7. Consider a linear time-invariant system, x = Ax + Bu, supposed 
controllable. The matrix L of the state-feedback control law u = — Lx which 
shifts the eigenvalues of this system towards desired values X Lj , i = \,...,n , is 

given by 

L = PD"Vo(A) , (1.75) 
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D = Q C 



fP<7i(A ca Jl 
P<7 2 (A ca J 



with Q c = B AB 



A" _1 B 



(1.76) 



and 



9o( A ) = A "+«»-i A " l + --- + a l \ + a (s ={\-\ Ll )...{\-\ Ln ) 
q l (X) = \"~ +a n _- l X"~ +... + a 2 \ + a l 



h (X) = \"- i + a n _ i X"- i - l + ... + a i 



?„(A) = 1 



(1.77) 



and where A can is the closed-loop system matrix, chosen in one of the known ca- 
nonical forms, e.g. the controllability form, and having the desired eigenvalues 
X Li , i = l,...,n. The elements of the (pxn) matrix P are the parameters which 

can be chosen arbitrarily in a MIMO pole-placement design, with the only restric- 
tion that D in (1.76) be invertible. 

Proof. By means of a regular transformation x = Vz , the closed-loop system ma- 
trix 



A CL = A BL 



(1.78) 



is transformed into a canonical form A can , which is desired to have the prescribed 

eigenvalues X Li . 

Right multiplying (1.78) by V and letting 



yields 



P = LV 



A CL V = AV BLV = AV - BP . 



(1.79) 



Since A can = V A CL V (equation (A.30), Appendix A), it follows from the pre- 
vious equation that 



AV-VA Cfl „=BP 



(1.80) 



IS 
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The transposition of this equation, 



V T A T -AjV T = P T B T , 



makes it appear formally identical to the equation 

TA FT = GC, 



(1.81) 



which is encountered in the observer theory (see Sect. 2.2.4.6), by introducing the 
following duality: 



Observer Theory - 


-» Control Theory 


T 


-> V T 


A 


-> A T 


F 


- Aj 


G 


-> P T 


C 


- B T 



(1.82) 



This remarkable duality between control and observation (or estimation) will 
be reviewed in more details in Chapter 2. 

In the case of Equation (1.81), where the matrices A (nxn), F (rxr) , G 

(r x q) and C (qxn) are supposed known and where the unknown is the matrix 

T (rx«), [CeBa84] has shown, building on the works of [B0Y068], that, if the 

eigenvalues of F are all different from those of A, this equation has a unique solu- 
tion T, given by 



T= G FG F"G 



(C ?1 (A)} 

Cq 2 (A) 

[Cq r (A) 



'(A), 



where the polynomials <7,(A) are given by (1.77) by letting n = r . The applica- 
tion of the previous duality to this result, with r = n , yields 



pT A T pT 

r A can r 



{Kan) P' 



B T ^(A T ) 

B T <? 2 (A T ) 



IB T <7„(A T )J 



<7o -1 (A T ). 
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V = 9o - 1 (A)D, 
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(1.83) 



where 



D-( ft (A)B <? 2 (A)B 

By developing the expression of D, 

D = < ?1 (A)BP + < ?2 (A)BPA ca „ + ... 



*»(A)B) 



P 
PA a 

pa"; 



-q n _ l (A)BVA:; n z + q n (A)BVA:;:, 



then expressing the polynomials <7,(A) with the use of (1.77), and finally group- 
ing together the terms of same power of A, it is readily seen that 



D = BP< ?1 (A ca „) + ABP< ?2 (A ca J+. 

' p ?i( A ca «y 

P?2(A ca J 
p 9„(A ca „)J 



- A"" 2 BP q n _ x (A can ) + A-'BP q„ (A can ) 



B AB 



A" "B 



Qc 



fP9i(A ca „)l 
P9 2 (A ca J 



This expression is nothing else but (1.76). Finally, (1.79) and (1.83) yield 

L = PV- 1 =PD" 1 9o (A), 

which proves (1.75). 

Remark 1.16. Similarities with the complete modal design. 

Though the present approach is entirely different, a strong similarity is notice- 
able with the expressions of the complete modal control design. Compare in par- 
ticular relations (1.79) and (1.58): the parameter matrix P plays here the same role 
as the matrix P = (pj ••• p B ) of the parameter vectors introduced by Rop- 

penecker. 

Here also, the matrix P is invariant in a similitude transformation. Furthermore, 
(1 .79) brings about that u = — Lx = — LVz = — Pz . 

Finally, if the plant eigenvalues and those prescribed for the closed-loop system 
are all distinct from one another, and if the diagonal canonical form is chosen for 
A can , the Roppenecker's formula appears as a particular case of the present ap- 
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proach. Indeed, the transformation matrix V is in this case the modal matrix of the 



closed-loop system, V = (v z 



v L „ ) , which is composed of its eigenvectors, 



so that (1.79) becomes identical to (1.61) and (1.75) to (1.63). All the remarks of 
Sect. 1.6 apply thus in this case, in particular those of Sect. 1.6.2 on the targeted 
choices of the parameter vectors and of the eigenvectors. 

Remark 1.17. Differences with the complete modal design. 

If for A can the controllability canonical form is chosen, the necessity, which 

appears in the complete modal design, to distinguish between the case of all dis- 
tinct closed-loop eigenvalues and the one of multiple eigenvalues disappears here 
[BeOs87]. 

Also, the limitation of the Roppenecker's formula to a multiplicity order of 
multiple eigenvalues inferior or equal to the number of plant inputs disappears 
here: any number of identical eigenvalues can be imposed to the closed-loop sys- 
tem if A can is chosen in controllability canonical form. 



1. 7.2 Single-input Systems (p = 1): Retrieving 
Ackermann 's Formula 

The parameter matrix P, of size (1 x «), becomes then a row vector, 

p = p t = (a - P„). 

Starting from (1.76), the initial plant being assumed controllable, which has the 
consequence in this single-input case that the matrix Q^ 1 exists, we can write that 



PD _1 =p T 



P T <?2 ( A ca«) 



J? VniKan), 



Qc 



Taking into account that 



P T fc( A «J 

P T <72( A «m) 



P <ll(Kan) 
P T 1l( A can) 



(1 ••• 



o) 



J? (IniKan), 



J? <l„( A can), 



••• 
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and that q n (A can ) = 1 by definition, we deduce from the last line of the above ma- 
trix equation that 



P T <?2 (Kan) 



J? VniKan), 



1 . 



This leads easily to 

PD-^O - \)Qi?=<&, 
and the control law (1.75) becomes 

L = qc?o( A )- 

Since qj is the last row of the controllability matrix inverse, and since q (X) 

is, according to (1.77), the desired characteristic polynomial of the closed-loop 
system, we recognize here the Ackermann's formula (1.27). 

1.8 State Feedback with Integral Action 

It has been seen previously that the static unity gain of a state-feedback control 
system was ensured by the gain compensation (feedforward) matrix M. This solu- 
tion has of course the inconvenient not to guarantee that the steady-state error van- 
ishes exactly, especially if the plant model is not perfectly known. 

An alternative consists in adding one or more integrators to the control loop, 
exactly as is done for the design of controllers including an integral term (I-term) 
in the case of SISO systems. 

1.8.1 Continuous Case 



In order to cancel the steady-state error, let us add q integrators at the output of the 
comparator, one by reference vector y r component, as for the design of cascade 

controllers in the case of SISO systems, and feed them back by means of a matrix 
L 2 , the state vector x of the plant being fed back as previously by a feedback ma- 
trix denoted here by Lj . 
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The control system has then the block diagram of Fig. 1.7: 



OH 



Q~ 



<Dr 



y 



Fig. 1.7 Continuous- time state-feedback control with integral action. 

The state vector T| , of size q, reflects the addition of the q integrators (one ad- 
ditional state variable per integrator output). The plant state equations are 



x = Ax + Bu 

y = cx 



As to the additional state variables, they are described by 

T| = y r -y = y r -Cx. 



(1.84) 



By grouping these q state variables together with the n state variables describ- 
ing the plant, we obtain the augmented system described below. 



1.8.1.1 Augmented System 

It is represented by the differential state equation 

x = Ax + Bu 
Tl = -Cx + y,. 

thus by the following augmented state representation: 



X 


= 


A <T 
-C 0, 


X 


+ 


B 

/ 


u + 




y =(c o)f 3 


s 






I 1 


lj 











(1.85) 



(1.86) 
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The state-feedback control law introduced previously (see Fig. 1.7) is of the 
form 



u= LjX — L 2 Tl = — (Lj L 2 



The augmented system closed by state feedback obeys then the differential 
state equation 



f A 0) 
-C 



L, L 2 



A-BL, BL 2 



x 



0) 



'0l 



y r 



Yr: 



c o 



(1.87) 



1.8.1.2 State Feedback Design 

The augmented system (1.86), of size (n + q) , being described by 



X 


= A 


X 


+ Bu + 


0' 


y 


= C 


X 


•> 






J\ 







where 



'A 0^ 


, B = 


B 


-C 0, 








and C= C 0) 



the present problem consists in determining the augmented state-feedback matrix 

£ = (1^ L 2 ) (1.88) 

so that the closed-loop system, resulting from the control law 



x • 
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and having the system matrix F = A — BL , has the desired dynamic behavior. 

The design can then occur by one of the methods described previously, e.g. by 
assignment of the (n + q) poles or by any of the other methods presented in Sec- 
tions 1.4 through 1.7, provided the pair (A, B) is controllable. [DucOl] has 
shown that this condition is equivalent to the three following: 



1 . rank 



A B 

= n + q ; 

C oj 

2. the pair (A, B) is controllable; 

3- P>q, i.e. the number of control inputs must be at least equal to the number of 
outputs to control. 

1.8.1.3 Cancellation of the Steady-state Error with Respect to 
the Reference at Equilibrium 

Let us verify the efficiency of integrator introduction into the control loop by cal- 
culating the permanent error of order zero, or static error, at the equilibrium state, 
y — y r , for a constant reference signal y r . The equilibrium state of the augmented 

system is characterized by 



x 



0. (1.89) 



The second of these equations, i\ = , leads, according to (1.84) or (1.85), to 

-Cx + y r =0, 
thus y = Cx = y r . 

1.8.1.4 Elimination of Constant Disturbances at Equilibrium 

The addition of integrators in the control loop should also cause the cancellation 
of a steady-state error stemming from the application of a constant measurement 
disturbance or a constant load disturbance, since the integrators are placed be- 
tween the comparator output and the point of application of these disturbances. 

Let us verify this property. 

The previous diagram is completed, as shown in Fig. 1.8, by a constant meas- 
urement disturbance w, of size q, and by a constant load disturbance v, the size of 
which is unspecified but is taken into account by the matrix E so that it can be 
added to the state equation input. 
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Fig. 1.8 State-feedback control with integral action and load and measurement disturbances. 

The augmented system, completed this way, obeys the following state equa- 
tions: 
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The closed-loop differential state equation (1.87) is replaced here by 
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Equilibrium state. The condition (1.89) which characterizes the equilibrium state 
is equivalent to the two following conditions: 

1. x = , which leads to x ^A-BLj)" 1 (BL^-Ev^O , thus also toCx * 

2. fj = , which leads to — Cx + y r — w = , thus y = Cx + w = y r . 

The constant disturbances are therefore eliminated at equilibrium state. 



1.8.2 Discrete Case 



The diagram of a discrete system equipped with q discrete integrators, added at the 
comparator output for the same reasons as in the continuous case studied previ- 
ously, is shown in Fig. 1.9. 
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x t+l I— x k 




Fig. 1.9 Discrete-time state-feedback control, with q discrete integrators. 



1.8.2.1 "I" Controller 

The addition of a discrete integrator at the comparator output reflects itself in the 
difference equation 

i|*+i = ii* + y,,*-y*> 

where the integrator output, x\ , represents q additional state variables, which add 
to the n state variables describing the plant. 



1.8.2.2 Augmented System State Equations 
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1.8.2.3 State Feedback Control 


UA-=- L i x /t- L 2 r U=-( L i L 2 ; 
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1.8.2.4 Static Error at Equilibrium 
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The equilibrium state is characterized here by r\ k+1 = r\ k , thus, according to the 
second equation of (1.91), = -Cx /t +y r k , i.e. y t =y rt . 



1.8.3 Partial Integral Action (Continuous or Discrete) 

In the two previous sections, it was assumed that the output vector was fed back 
entirely, by means of one or more integrators introduced in the control loop. 

It may happen that this "I-type" control is wished for only part of the measured 
quantities, i.e. those for which exact cancellation of the steady-state error is most 
critical. This limitation can stem from, e.g., the wish not to increase too much the 
number of state variables, in the case of a large size MIMO system, or not to pe- 
nalize too much its transient response by the addition of the corresponding poles. 

The choice of the q t components of the output vector y(t) or y k which one 

wants to feed back by integrators is made by a selection matrix S ; , as illustrated in 
the two universal simulation diagrams of Appendix D, so that the partial output 
vector which is fed back is given by 

y,=S ! .y = S ! .C m x = C,x, 



with C, = S,C m . Even though the method applies equally to the discrete case, the 
following calculations will be developed only in the continuous case. If x\ , of size 
q i , denotes as previously the vector of added state variables, the open-loop aug- 
mented-system state equation (1.86) becomes 



A 
C, 



(0) 

s 



Yr: 



its output equation remaining unchanged. 

The application to this system of the control law illustrated in the simulation 
diagram and including the gain compensation resulting from the matrix M, or 
S C M if p<q as discussed in Sect. 1.1.3.2, 



L, L 2 



■My r) 



yields for the closed-loop system the following state equation: 
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-C, 
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Determination and role of the M matrix. In steady state, the response to a con- 
stant reference y r gives then 

= (A-BL,)s„- BL 2 n „ + BMy r , 

where x^ and n „„ denote the state and its augmented part at equilibrium. Accord- 
ingly, the output at equilibrium has the following expression: 

y M =Cx oo = C(BL t AV^BL^ +C(BL! -A) -1 BMy, . 

Let us write, similarly to (1.8), M = /ofcCBL, - A) _1 BJ , with < p < 1 , re- 
spectively M = pS c [c(BLj - A) _1 BJ "' if p <q . 

Due to the presence of the integrators, n^ is adjusted automatically so that 
S, y TC = S, y r . In the case where p = 1 , situation which has been chosen in the ac- 
companying software MMCE.m, the equality y oo =y r , which leads to 
S, y TC = S, y r , is obtained for n^ = . The anticipation term, or feedforward term, 
constituted by the M matrix yields then at equilibrium the maximum contribution 
to the required control, the integrators having no effect anymore, in the hypothesis 
of a perfect model and in the absence of any disturbance. Due to its immediate ap- 
plication after a step reference, this term accelerates the transient response. 

In counterpart, it causes generally also overshoots of this response, due to the 
zero(s) that it introduces into the closed loop, exactly as does a feedforward com- 
pensation term in SISO control loops. 

Its influence can be reduced by choosing p < 1 , but in this case the equality 

y M = y r can no longer be obtained, unless C, = C , i.e. S, = \ q . 

An application is proposed in the frame of the solved exercise dealing with the 
control of a magnetic tape drive, at the end of Chapter 4. 



1.9 Solved Exercises 

Preliminaries and Computer-based Calculations. Most of the exercises pro- 
posed in this book require the use of digital tools to be solved. Due to the impor- 



1 .9 Solved Exercises 59 

tance and universal acceptance of the MATLAB® 2 software environment in this 
field, a generic program, created under MATLAB 7.9 (R2009b) but compatible 
with earlier versions and covering the five plants which will be used in the exer- 
cises, is freely available to the reader. It contains a main program MMCE.m and 
several subprograms, bound in the archive mmce.zip, which is downloadable on 
Internet from the Springer or The MathWorks sites: 

http://extras.springer.com/2010/978-3-642-13733-4 
http://www. mathworks. com/matlabcentral/fileexchange . 
It enables the reader, eager to solve the exercises by himself, to compare his solu- 
tions with the given ones, and even to find other solutions, knowing the multiplic- 
ity of choices offered in MIMO control. The limited place available in this book 
has indeed not made it possible to apply them all on each of the included plants. 

Simulation makes this learning process even more vivid, and the reader is never 
advised enough to undertake simulations of his solutions. Three Simulink® simu- 
lation diagrams, which enable repeating the simulations of about all the exercises, 
are given in Appendix D. They can also be used as templates to create other dia- 
grams, at the reader's liking. 

Despite all the care brought to establishing software modules or diagrams, no 
warranty can be given as to the absence of any error. It is therefore called upon the 
reader's perspicacity in his use of these tools. 



Exercise 1. 1 Control of an Inverted Pendulum 

The plant to study is sketched in Fig. 1.10. It represents a real experimental setup, 
which will be used repeatedly along this book [Ami92]. Thus the real parameters 
of this setup will be used in the present exercise as well as in the followings, and 
their values appear in the generic script /w-file MMCE.m, under the plant name In- 
verted pendulum LIP 100 (Amira). 

The pendulum rotates about an axis, carried by a cart, which is moved horizon- 
tally by a belt and a motor, not shown in the figure, which apply a force F to it. 

The mathematical representation of such a system has been established in sev- 
eral books, typically: in [MoMa98] by the method of the Lagrangian equations, 
described e.g. in [Ost04], by neglecting all the frictions; in [FrPE94] where the 
viscous friction of the cart on its rail is taken into account by writing the equilib- 
rium of forces and the fundamental equation of dynamics; in [Ami92] where the 
viscous friction of both the translation movement and the rotation movement are 
taken into account, also by application of the fundamental principle of dynamics 
and the balance of forces involved. It is this last situation which will be considered 
here. 



2 Distributed by The MathWorks. The only toolbox required to work out the exercises is the 
Control System Toolbox. To do the proposed simulations, it is necessary to have also Simulink . 
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x c : cart position 
0: pendulum angle 
F : force applied to the cart 



x c x 

Fig. 1.10 Inverted pendulum on a cart. 

The model equations, linearized around its unstable equilibrium point in the 
small angle approximation, are the following 3 : 



(M c + M p ) x c +M/8+ B c x c = F 
\(J + M/ ) 9 + M/ x c + B p 0-M/g9 = O 

where the parameters have the following meanings: 

M c , M = masses of the cart and the cylinder mounted at the rod extremity; 

B C ,B — viscous friction coefficients of the linear cart movement and the rota- 
tional rod movement; 

£,J= length and moment of inertia of the rod. 

Choosing the following state variables and input quantity: 

and expressing all physical quantities in terms of their electric tensions, delivered 
by the sensors or applied to the electronic circuit driving the motor, the state 
model of the plant [Ami92] in continuous time is given by 



A = 









-1.950 







' 







-0.1289 



-1.915 


1 
0.00082 


; b = 



-6.1344 





21.47 


26.34 


-0.1362 




84.303 



This plant is sampled at a period T s =0.03 s , and the discrete model is given by 



3 In certain books it is the positive trigonometric direction which is used as positive angle direc- 
tion, and not the clockwise direction as is the case here. It is enough then to inverse the sign of 
all terms in 9 , 9 or 9 . 
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The state variables x l and x 2 will be selected here as output quantities, al- 
though x 3 (but not x 4 ) is also measurable. The measurement matrix is therefore 



C = 



10 
10 



The notation C m will be used in all this book's exercises solved by MMCE.m 
for the matrix building the real measurements. An identity matrix C is indeed in- 
troduced in the simulation diagrams inside the blocks representing the plants in 
state-space representation, so as to give full access to their state vector. 

The reader is free to modify this measurement matrix C m at will, in order to 
perform other tests. 

Taking into account nonlinear phenomena. The most significant nonlinearities 
are the dry friction force (Coulomb friction, of intensity F c ) to which the cart is 
submitted during its movement on the rail, and the stiction force (static friction, of 
intensity F s ) which appears at zero speed. The total friction force F? which coun- 
teracts the driving force, F d , as sketched below, is thus given by: 



f s ig n (-^ ) • min(| F d \ , F s ), for x = 0, 
[F c sign [x(t)j , for x * . 



a- 



In this case a compensation can be applied by superimposing an additional in- 
put tension which amounts to w s0 = Fj- jK F , where K F is the conversion factor 

expressed in N/V. The quantity u s0 can be either evaluated once and for all by 

measurements on the real set-up, or estimated in real time by means of a distur- 
bance observer. This second method will be the subject of an exercise in Chap. 2. 

In the present exercise, we will not take into account this nonlinearity and will 
consider the plant as a linear system corresponding to the above state space model. 
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This exercise will be solved in the MATLAB® environment, with the MMCE.m 
program mentioned above. 

a) Determine the pole-zero map of the open loop. 

b) It is desired to stabilize the pendulum in its upright position, the cart being 
stopped at a reference position y = x l = y r on its rail. After having 

checked the controllability of the system, design a state-feedback control 
for it by the pole placement method, by assuming first that the state vector 
is fully accessible and by imposing to the closed-loop system the following 
multiple eigenvalue: \ Ll ■ = — 5 s _1 , i = 1, • • • , 4 . 
Determine the poles and zeros of the closed-loop system and discuss. 

c) Simulate this control system and log its step response. Observe in particular 
the steady-state values. 

d) Switch to the discrete model and repeat questions b) and c). 

Solution: 

Run the generic program MMCE.m and select the plant inverted pendulum 
LIP1 00 (Amira) with two measurements. 

(a) The statement [Poles Zeros] = pzmap(A,B,Cm(1,:),D(1,:)) included in 
the software yields the following values for the open-loop plant, from its unique 
input u to its output y = x i =x c : 

poles: s 1 =Q, s 2 =4.51, s 3 =-4.83, s 4 =-1.73 ; zeros: z 1= -4.50, z 2 =4.38. 

(b) In the program, select then successively: Choice of model: Continuous 
(default), Synthesis of state-feedback (default), Integral action: = without 
(default), Component of y to be set to its reference value: 1 , then Algorithm 
to be used: 1) pole placement (MATLAB modules « acker.m » or 
« place. m »). 

Before the last prompt of the program, the rank of the controllability matrix 

Q c =(b Ab A 2 b A 3 b) has been computed by the statement 

rank(ctrb(A,B)): it is equal to 4, thus to the order n of the plant, which is therefore 
controllable. 

Since the plant has less control inputs than measurement outputs (p<q), the 

program prompts the user to select the output or outputs which should be regu- 
lated without steady-state error to a constant input reference. Answer here: 1, so as 
to obtain a unit static gain on the position output ( y = y { = x l ). 



1 .9 Solved Exercises 



63 



Since we have entered a multiple eigenvalue with a multiplicity order greater 
than the number of control inputs, the algorithm acker.m (Ackermann's method) is 
chosen automatically by the program. The obtained result is: 

L = £ T = (-2.652 2.449 4.485 0.539) 
M = -2.652 

The closed-loop poles and transmission zeros from y r to y = x l are found by 
executing the statement pzmap(sys_CL), the closed-loop system being built by 
sys_CL = ss(A_CL, B_CL, Cm(1:p,:),D(1:p,:)), with A CL and B Ci given by 

(1.4) and (1.5): 



poles: 5j = —5.0008, s 2 ; 
zeros: z, = -4.50, z 2 = 4.38 . 



-5.0000 ±y'0.0008, s 4 =-4.9992; 



The poles are at the prescribed locations, but a small inaccuracy is noticeable, 
which is characteristic of Ackermann's method, in particular in the case of high 
order plants and multiple poles, which is the case of the present selection. The 
place, m algorithm also proposed by the program is more robust, but it does not al- 
low entering multiple eigenvalues with a multiplicity order greater than the size of 
the input signal (equal to 1 here, the control u being scalar). 

The zeros remain unchanged as we have seen in Sect. 1.2.4. The existence of a 
zero in the right-half complex plane, often also called improperly unstable zero 
though it does not have any effect on the system stability, reveals on the contrary 
the nonminimal phase characteristic of the plant, as well in open loop as in closed 
loop. 

(c) The closed-loop step response is plotted by using the statements at the end 



of the program and is reproduced in 
the oscillogram to the right. The reader 
having the Simulink® software at his 
disposal can retrieve this response also 
by means of the generic simulation 
diagram given in Appendix D. The S c 
matrix of this diagram extracts from 
the reference y,. = [l 0] the unique 
component y r l for which the M ma- 
trix (a scalar, here) can ensure a unit 
static gain (see the third paragraph of 
the answer to question (b) above and 
the discussion of Sect. 1.1.3.2, Case 
where p ** q). 
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In the transient part of the output y x step response we observe an undershoot, 

which is characteristic of nonminimal phase systems. The steady-state response 
corresponds to a unit static gain, which is obtained by the gain compensation, or 
feedforward, matrix M, which is here a scalar. 

(d) Select now the option Choice of model: discrete, and proceed as previ- 
ously. The closed-loop eigenvalues to enter are the continuous-time values, the 
program converting them automatically to their discrete counterpart at the plant 
sampling time, here T s = 0.03 s . The obtained discrete control law is 

£ T = (-2.0298 2.0162 3.5714 0.4434); M= -2.0298. 



Closed-loop poles and zeros: p t = 0.8607 = e 



-0.03x5 



1, 



z 1= -0.9809, z 2 = 0.8737, z 3 = 1.1403 

We note the presence of an additional zero as compared with the continuous case: 
it results from the sampling of the continuous plant with zero-order hold. 
The step responses are identical to the previous ones, at the sampling times. 



Exercise 1.2 Control of a Three-tank System 



The considered MIMO plant is sketched in Fig. l.ll 4 .The state variables x x , x 2 
and x 3 , respectively the liquid feeding flows u x and u 2 represent the deviations 

of the three liquid levels from their reference heights, respectively from their ref- 
erence flow rates, which correspond themselves to a given equilibrium state. 

If we assume that only the level deviations x x and x 3 are measured, the state 
representation of this plant is 




Fig. 1.11 Three-tank system. 



1 Numerical example introduced in [F6190] and reused in [Rop90] . 
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A = 



C = 



0.332 0.332 

0.332 -0.664 

0.332 

(l 0) 
1 



0) 
0.332 
-0.524 



B 



'0.764 














0.764 



the liquid heights being expressed in meters and the flow rates in m / mn . The 
symmetry of matrices A and B is a consequence of the equality of the tank and 
pipe cross areas. 

a) Calculate by the simple modal control method available in the program a 
continuous-time state feedback which shifts the two eigenvalues corre- 
sponding to the two slowest modes to the following values: 
— 0.5mn _1 ; — 2mn _1 . 
Plot or observe by simulation the free response of the three state variables, 



starting from an initial state x 



-0.8 -0.5 -0.2) , for the open-loop 



plant, then for the system closed by the controller calculated in (a). 

b) Plot or observe by simulation the four individual step responses from input 
i to output j, for i, j = 1, 2 . In order to better separate the steady-state re- 
sponses, steps of different amplitudes, different from unity, will be applied 
by the program to the two inputs: y r — 0.4 and y r = 0.8 . What do you ob- 
serve? 

c) Repeat this design by using now the Falb-Wolovich input-output de- 
coupling algorithm, and compare the resulting step responses with those of 
question (b). If possible, use the same eigenvalues as imposed in (a). 

d) Determine the zeros of the closed-loop system, and compare them with its 
eigenvalues. 

Solution: 

Run the generic program MMCE.m by choosing the three-tank system plant. 

(a) The program calculates the initial plant eigenvalues: 

A 1= -1.0359, A 2 = -0.4374, A 3 = -0.0467. 

Since the plant has two control inputs, only two eigenvalues can be given new val- 
ues by this method. It is then usual to choose the slowest ones, so we decide to 
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place A 2 at \ L2 = —2 and A 3 at A i3 = —0.5 
achieves this is then 



1.0717 
-0.8003 



0.1280 
0.6388 
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The state feedback matrix which 



-0.5820) 
1.5670 



The free response, or return to equilibrium at vanishing input from a non van- 
ishing initial state, x = (—0.8 —0.5 —0.2) , is plotted in Fig. 1.12, for the ini- 
tial plant and the closed loop. The two time scales have been chosen different so 
as to enlighten better the time reduction achieved by the regulation in eliminating 
the initial disturbance. 

A verification by simulation shows that the two control signals applied to the 
plant inputs during the transient response do not exceed 1 m / mn , which repre- 
sents the maximum supposed flow rate of the actuators in this example. If this 
were not the case, we should choose closed-loop eigenvalues closer to the imagi- 
nary axis, which would of course result in a slower transient response. 
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Fig. 1.12 Free responses of the state variables: (a) open-loop; (b) closed-loop. 

(b) The step responses to the references given in the problem statement are 
plotted in Fig. 1.13. 

Thanks to the feedforward matrix M which has been calculated by the program 
at question (a), 



M = 



1.3529 -0.7353) 
-0.6981 2.3550 



a steady-state unit gain is ensured between each of the references and the corre- 
sponding output. As to the transient behavior, the signal applied to input 1 has a 
visible influence on output 2 and vice-versa. This reveals a coupling inside the 
plant. 
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Fig. 1,13 Step responses of the closed-loop system. 

(c) The design occurs here by the first of the two methods available in the pro- 
gram under decoupling method (Falb-Wolovich or Roppenecker). 

The order differences of the two outputs, 8 X and 6 2 , are both equal to 1, since 



c 1 r B = (l 0)B = (0.764 0)^0 
cJB = (0 1)B = (0 0.764)^0. 



The total order difference of the plant amounts therefore to S = S 1 + S 2 = 2 , and 

only 2 eigenvalues can be imposed to the closed loop. Let us use the same eigen- 
values as in question (a), for comparison purposes: —2 and —0.5 . The following 
control law and feedforward matrices are obtained here: 



'2.1832 


0.4346 


0' 


; M = 


'2.6178 








0.4346 


-0.0314 







0.6545 



(1.92) 



The step responses are given in Fig. 1.14. One will notice that the couplings have 
disappeared, each of the reference signals acting only upon its associated output. 
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The closed-loop system is decoupled, as confirmed by its transfer matrix G(s) , 
calculated by the statement Gw = tf(sys_CL) of the program: 



G(s) = 



s + 2 





0.5 



s + 0.5 ) 



Among its poles are as expected the two eigenvalues imposed during the design, 
from u x ... from u 2 ... 
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Fig. 1.14 Closed-loop step responses with decoupling control. 

(d) The closed-loop system has one zero, given by Z = zero(sys_CL): 
z, =-0.664. 

Its eigenvalues, given by eig(sys_CL), are: —2 , —0.5 and —0.664 . It appears 
thus that the third closed-loop eigenvalue has been placed by this design exactly at 
the location of the zero of the system, open or closed-loop since it is the same, 
which has had the effect to cancel out this pole from the transfer matrix of the de- 
coupled system. 
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This is confirmed by using the statement Msys_CL = minreal(sys_CL) of the 
program, which answers by indicating that one state has been removed. This 
statement builds indeed the minimal representation of a system by suppressing all 
uncontrollable and/or unobservable modes (see Sections A.2. 3 and A.3.3 of Ap- 
pendix A). 



Exercise 1.3 Complete Modal Control of a Three-tank 
System 

Consider again the three-tank system of the previous exercise. 

a) Take advantage of the degrees of freedom offered by the complete modal 
control to realize a state-feedback control which decouples the inputs- 
outputs. Comment the differences with the Falb-Wolovich method. 

b) The middle tank has now a hole through which a constant liquid flow leaks, 
with a flow rate of 0.5 m 3 /mn. By considering this flow as a constant dis- 
turbance v, check if it is possible to calculate a state-feedback control 
which decouples this disturbance from the outputs, and, if the answer is 
positive, observe by simulation the effect of the disturbance on the closed- 
loop system outputs. 

Solution: 

Run the MMCE.m program by selecting the synthesis of a state feedback by 
the decoupling method and choosing this time its second variant (method of 
Roppenecker). 

(a) As has been seen in Sect. 1.6.5, the choice of the eigenvalues and the pa- 
rameter vectors is not arbitrary, if one wants a decoupling of the closed-loop in- 
puts and outputs. These choices are going to be applied in this module. The first 
step consists in looking for eventual zero or zeros of the initial plant. The state- 
ment [Poles Zeros] = pzmap(Ac, Be, Cmm(1:p,:), D(1:p,:)) of the main pro- 
gram has given a zero at \x = —0.664 . One eigenvalue must then necessarily be 

placed at X L1 = —0.664 : this is done automatically by the subprogram 
Falb_Wolovich_or_Roppenecker.m. One chooses then freely \ L2 =—2 and 
A i3 =—0.5 so as to provide for comparison of the results with those of Exercise 
1.2. 

The program determines then the directions associated with this zero, i.e. the 
non vanishing solutions of the homogeneous equation constructed with the Rosen- 
brock matrix (Appendix A, Sect. A. 6): 
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'A -jil B 
C D 



The five statements Rosen = [A-Z_OL(i)*eye(n),B;C,D], nsp = null(Rosen,'r'), 
nv = size(nsp,2), Xz = nsp(1 :n,1 :nv), and Uz = nsp(n+1 :n+p,1 :nv) yield 



x 7 =(0 -2.3012 Of and u 7 =fl if. 



which determine the eigenvector and the parameter vector associated with the ei- 
genvalue X Li = (i , according to Step 1 of the procedure of Sect. 1.6.5: v L1 =x z 
and P[ = — u z . The two other parameter vectors are calculated according to (1.71) 



p : =(2.0753 -0.10799) , p 2 = (0.8797 0.8483) . 

The final result is exactly the same control law (1.92) as was obtained by the 
method of Falb-Wolovich used in Exercise 1.2, question (c). 

There is thus no difference between these two designs, in the present case. If, 
on the contrary, the plant zero had been situated in the right -half plane, the algo- 
rithm 3) complete modal synthesis (Roppenecker's formula) of the program 
would have given us the choice not to compensate it by an imposed eigenvalue, 
and to perform only a partial decoupling, what the Falb-Wolovich method does 
not allow. 

(b) The second tank leak represents a constant load disturbance v applied to the 
plant by means of the vector e = (0 1 0) . The feasibility necessary condition 
(1.73) of such a synthesis, namely 



C ra e = 



1 0" 


(0 1 o) T = 


'0 


1 






is satisfied. Furthermore, the plant has indeed a zero and the input vector e can be 
written as a multiple of the state direction associated with this zero, 

x z =(0 —2.3012 0) . The envisioned synthesis is thus possible. One eigen- 
value must be placed at the location of the zero and the corresponding parameter 
vector must be chosen equal to the direction of the control vector associated with 
this zero. The two other pairs of eigenvalues and parameter vectors can be chosen 
according to other criteria. 

Since the design of the L matrix performed at question (a) fulfills precisely 
these various requirements, the control law obtained in that question can be used 
here. The oscillograms of Fig. 1.15 represent the step responses, with the applica- 
tion of the disturbance having a 0.5 m 3 /mn amplitude at time t = 15 mn . By com- 
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parison, the same responses are plotted for a complete modal synthesis (Rop- 
penecker's formula), performed with the same eigenvalue choice as in question 
(a), including the mandatory choice of \ u =—0.664 which is not done automati- 
cally here by the program, but with randomly chosen parameter vectors: (l 2) , 
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Fig. 1.15 Simulation oscillograms obtained with the 

(a) decoupling method according to Roppenecker; 

(b) complete modal synthesis (Roppenecker' s formula), with random parameter vectors. 

It appears clearly that in case (b), the disturbance-output decoupling is lost. 



Exercise 1.4 State Feedback with Integral Action 

The three-tank set-up of the two previous exercises is considered again, but the 
study will be made here in discrete time, with a sampling period of 0. 1 minute. 



a) Calculate for this plant a control law with integral action, which gives the 
closed-loop system the same continuous-time eigenvalues as in Exercise 
1.2, plus additional eigenvalues to choose judiciously. Choose freely one of 
the available design methods. The simple modal control is not usable in the 
present case. Indeed, the plant having only 2 inputs, only 2 eigenvalues can 
be shifted by this method. Now, in the continuous as well as in the discrete 
case, the integral action adds two unstable poles to the open loop of the 
augmented system, respectively at s = or at z = 1 . One would thus have 
during the design process to shift only these two eigenvalues, which has 
obviously no interest. For similar reasons, the decoupling control is no 
longer useable either. 
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b) Study by simulation the influence of load disturbances and of measurement 
disturbances on the closed-loop response. 

Solution: 

(a) Run the MMCE.m program, with the same plant as previously, by selecting 
Choice of model: D) discrete, then a synthesis of A) state feedback and In- 
tegral action: 1 (= with). Since the program allows partial integral feedback (see 
Sect. 1.8.3), it prompts the user to indicate, in the form of a row vector, the indi- 
ces of y components to feed back by integrators. Answer here: (1 2), i.e. 

total integral action, on the two measurements. 

The program has created the following augmented model (Sect. 1.8.2.3, equa- 
tion (1.90)): 



O 



0.9679 0.0316 0.0005 

0.0316 0.9368 0.0313 

0.0005 0.0313 0.9495 

-10 10 

-10 1 



(0.0752 

0.0012 0.0012 

0.0744 







Design then the state feedback by pole placement, with the following closed- 
loop poles: -2, -0.5, -1.0359, -10 and -10, the first three poles being chosen so 
as to allow comparison with Exercise 1.2. The last two poles imposed to the 
closed-loop system come from the addition of the two integrators, and their choice 
was guided here by the only concern not to slow down too much the dynamic be- 
havior of the closed-loop system. We obtain: 



Lj 



M = 



' 9.1213 1.7650 
-0.3734 2.9596 


0.0312' 
10.1484 , 


, L 2 = 


-0.6869 
0.3012 


0.0163' 
-1.4152 , 


10.221 0.696' 












0.889 12.097 













Since we are dealing here with a multiple input system, the method of Acker- 
mann does not apply anymore, and the program uses here the place. m module. 
Values delivered by an earlier version of the MATLAB® software might differ 
slightly from these, due to the evolution of the algorithms over the successive ver- 
sions. The place, m module tries indeed to optimize the control system robustness 
against parameter variations. 

The M matrix calculated by the program is no longer indispensable here, the 
cancellation of the steady-state error being guaranteed by the two integrators. If its 
action is suppressed in the simulation diagram, the step responses are quite com- 
parable with those obtained in question (b) of Exercise 1.2. 
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If on the contrary this term is active, a strong acceleration of the transients is 
observed, but at the cost of important overshoots, of the order of 20%, as illus- 
trated in Fig. 1.16. The gain matrix M provides then the closed-loop system with 
an anticipation correction action (see Sect. 1.8.3). 

It is left to the reader to verify that with the other design methods roughly iden- 
tical results are obtained. 
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Fig. 1.16 Step responses with feedforward matrix M. 

(b) The effect of a load disturbance applied at t = 15 mn through the vector 

e = (0 1 0) T as in question (b) of the previous exercise, but with an amplitude 

0.5x7^ m 3 /mn in order to produce an effect comparable to the continuous case, 

is illustrated in the two oscillograms of Fig. 1.17, which have been logged in 
simulation without anticipation term. 
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Fig. 1.17 Output step responses, obtained with: 
(a) design of question (a); (b) complete modal design, with zero compensation. 



The left-hand oscillogram corresponds to the previous synthesis. The right- 
hand oscillogram has been obtained by performing again a complete modal de- 
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sign and by placing one of the two eigenvalues over the plant zero, thus with the 
following eigenvalues: -2, -0.5, -0.664, -10 and -10, the parameter vectors being 

chosen as follows: (1 0) T , (0 l) T , (l l)\ (l -2) T and (-1 0) T . As seen, 

the disturbance is again completely decoupled from the two outputs. 



Exercise 1.5 Lateral Motion Control of a Boeing 747 

The equations of the motion of a big airplane, such as a Boeing 747, are separated 
into two categories, which describe respectively the longitudinal and the lateral 
motions. It is this last movement which will be the subject of the present exercise. 
The lateral motion is composed of rolling (angle if , angular rate/?), of yawing 

(angular rate r) and of a side-slip, characterized by the side-slip angle [3 . Two ac- 
tuators allow stabilizing this movement and guiding the airplane along the desired 
trajectory: the angle 6a of the ailerons situated at the rear of the wings and the 
angle 6g of the rudder situated in the tail at the plane's rear. It is supposed that 

there are two sensors, one giving the yaw rate and the other the roll rate. 

One of the problems of blown wing aircrafts is to offer a very low damping for 
one of the lateral motion modes, as will appear below. One of the tasks of the con- 
troller to be implemented will be to increase this damping coefficient. 

In this exercise the following linearized continuous-time model will be used 5 : 
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In open loop, this plant has the following eigenvalues: 

• — 0.0329 ±7 0.9467 , complex conjugated pair which characterizes the very 
badly damped yaw motion, also called Dutch roll; 

• —0.0073 , which represents the very slow spiral mode; 

• —0.5627 , which represents the roll mode. 



5 The numerical values of this model are drawn from [FrPE94], p. 686, and have been completed 
by the author for the second control quantity by reasonable guess. 
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a) Calculate a state-feedback control law which imposes to the closed loop: 

2 eigenvalues \ n , X L2 =— 0.5 + y'1.0 affected to the side-slip mode, so 

as to make it faster and better damped than the open-loop Dutch roll 
mode; choose one of the components of the corresponding eigenvectors 

Vjj and v i2 such that the modes e u and e L1 do not appear on the 

roll angle; 

2 eigenvalues X Li = —2 and A L4 = —0.5 in order to accelerate the roll 
mode and the spiral mode; choose here also the corresponding eigenvec- 
tors v i3 and v L4 such that the modes e i3 and e i4 do not appear on 
the side-slip angle. 



b) Plot the free response of the side-slip angle and of the roll angle to the fol- 
lowing initial conditions: 0(0) = 10° = 0. 15 rad and tp(0) = 28° = 0.5 rad . 

Solution: 

(a) Run the MMCE.m program, by selecting the fourth plant, lateral motion of 
the Boeing 747, continuous model, then design a state-feedback control, 
without integral action, by the complete modal design (Roppenecker's for- 
mula), which allows influencing the choice of the closed-loop eigenvectors. 

After having opted in this module for the targeted choice of the parameter 
vectors, allocate the value to the fourth component of the two eigenvectors as- 
sociated with \ Ll and A i2 , and the arbitrary value 1 to one of the two components 
of the corresponding parameter vectors. Similarly, cancel the first component of 
the eigenvectors associated with A i3 and A L4 , and give again the arbitrary value 1 

to one of the two components of the associated parameter vectors. 
This yields the following control law: 



-20.1334 3.6117 13.7122 6.7387) 
5.0231 -1.2026 -3.3337 -1.6468 



and the closed-loop eigenvectors are: 



V = 



0.2028- y'0.0414 


0.2028 + y'0.0414 








0.0197- y'0.2224 


0.0197 + y'0.2224 


-0.0511 


0.0595 


0.0016 + 70.0179 


-0.0016 -y'0.0179 


-0.3752 


-26.3947 








0.1897 


52.7799 



(b) The free responses are given in Fig. 1.18: 
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Fig. 1.18 Free responses of: (a) the side-slip angle; (b) the roll angle. 



2 Observer Theory 



2.1 Introduction to the State Reconstruction Problem 

We have seen in the previous chapter that state feedback requires access to the 
state vector x(t) or x k , in its entirety. 

However, in many practical cases, even though some state variables will be ac- 
cessible to measurements, depending on the chosen state representation and the 
number and layout of the plant sensors, not all states will be measurable. 

It is therefore necessary to reconstruct the system state from what is available. 
Two points of view deserve then consideration. 



2.1.1 Purely Deterministic Point of View 

The following equations are at our disposal: 

Continuous Case: 
x=Ax+Bu or: 

y=Cx+Du 

The question which arises is then the following: why not deduce simply x 
from the second equation, x = C _1 (y — Du) , since y and u are accessible, one as 
output vector or measurement vector, the other as control vector fed at the input of 
the plant to be controlled? This is unfortunately impossible, for two reasons: 

1 . C is not in general a square matrix (q ^ n) , thus is not invertible; 

2. if there is additionally measurement noise (see following paragraph), this noise 
might disturb strongly such a reconstruction. 

Since an exact value of x(t) cannot be obtained, we will content ourselves with 
imagining a system which, from the known quantities y(t) and u(t) , will deliver 
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an approximate value x(t) , or estimate of x(t) . Such a system is called state re- 
constructor or observer. 

The remainder of this chapter deals with the design methods of such observers 
and their use for state-feedback control or the estimation of disturbances which 
can be modeled. 



2.1.2 Stochastic Context 

The previous problem complicates itself if the considered system is submitted to 
measurement noise and to random disturbances. 

The most general form of the state equations of such a system is: 



Continuous Case: 




Discrete Case: 


Jx = AX + Bu + EV 


or: 


jx i+I = *x t + ru t +Ev t 


[y = CX + Du + W 




[Y k -CX,+Du,+W, 



where: V represents the random disturbance vector applied to the state equation; 

W represents the random noise vector affecting the measurement. 
Note that the system state has become itself a random process X(/) in the con- 
tinuous-time case, a random sequence X k in the discrete-time case, on account of 
the presence of a random term in the state equation. Main concepts of random 
variables or vectors and random processes are summarized in Appendix C. 

Remark about notations. We have introduced here capital letters to denote ran- 
dom (scalar or vectorial) quantities, to align ourselves on the usual convention of 
Probability Theory, according to which a capital letter such as X (respectively, X) 
denotes a random variable (respectively, vector) and the corresponding lower case 
letter x (respectively, x) a particular deterministic realization of it. Knowing that 
there will appear no random matrices in the rest of this book, there should be no 
confusion for the reader with the previously adopted notation to represent matrices 
by bold capital letters. 

The state reconstruction problem enters here in the more general frame of what 
is called linear jiltering. 

Problem of linear filtering. Starting from the previously formulated situation, 
namely from the plant parameters (A, B, C, D or O , T , C, D) and from statisti- 
cal data about the noises V and W (statistical distribution, mean, variance), the 
problem consists in finding a causal linear system, at the input of which are ap- 
plied the signals accessible to measurement, i.e. u and Y, and which yields at its 

output a quantity X as close as possible to the unknown plant state X. 



2.2 Full-state, or nth-order (Luenberger) Observer 



79 



Such a system is called a filter (Fig. 2.1). The optimal solution of this problem, 

in the sense of the minimal variance of the estimation error X — X , is called Kal- 
man filter, and will be the subject of Chap. 4. 





Measurements: 
u 






Plant of 
unknown state X 


Filter 


x 


Y , 











Fig. 2.1 Problem of linear filtering. 



2.2 Full-state, or «th-order (Luenberger) Observer 



2.2.1 Basic Idea of the nth-order Observer 

To simplify the mathematical expressions, all the following theory will be devel- 
oped by assuming D = , a current situation in practice. One will find, if needed, 
formulae including the case where D ^ for instance in [LaTh77]. 



2.2.1.1 Continuous Case 



Consider the system 



x = Ax + Bu 

y = cx 



The basic idea of the Luenberger observer [Lue64] [Lue66] [Lue71] consists in 
adjoining to the above linear time-invariant system another linear time-invariant 
system, fed permanently by the known signals u(t) and y(t) , and which must de- 
liver at its output an approximate value x(t) of x(t) . 

The state differential equation of such a system will therefore have the following 
form, if one assumes, to make this system as simple as possible, that its output ma- 
trix C is the identity matrix I: 



x=Fx+Ju+Gy 



(2.1) 



where F, J and G are (nxn) , (nx p) and (n x q) matrices, to be determined. 
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While there is no reason to suppose that at t = the initial state of the system 
defined by equation (2.1), x = x(0) , is the same as the initial state x of the ob- 
served plant, it is on the contrary logic to require that x(t) — > x(t) for t — > oo • 
Introduce the estimation error 

x(t) = x(t)-x(t) . (2.2) 

x(t) will be a good estimate ofx(t) if x(t) >0 . 

In order to evaluate the repercussion of that requirement on the choice of the ma- 
trices F, J and G, let us establish the differential equation in x : 

x = x x = Ax + Bu Fx Ju Gy , 

or, substituting y = C x : 

x = Fx + (A F GC)x + (B J)u. 

The goal being to obtain that x(t) >0 , V x(^) and u(t) , one must choose 

J = B , and determine F and G so that A-F-GC = . 
The previous equation becomes then 

x = Fx. 

This homogeneous differential equation has, according to relation (A. 18) of Ap- 
pendix A, the general solution 

5(0 = e Ft x(0) . 

If the eigenvalues of F are at the left of the imaginary axis of the complex plane, 
x(t) -^0 for t — > oo , and the approximate value x(t) tends towards the true 
value x(t) . 

The system (2.1) defined this way is called a Luenberger observer, from its au- 
thor's name [Lue64]. 

Definition 2.1. A full-state or Luenberger observer of the system 

x = Ax + B u 

y = cx 



2.2 Full-state, or nth-order (Luenberger) Observer 

is a system described by the state differential equation 

x=Fx+Bu+Gy , 
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(2.3) 



where the eigenvalues of the F matrix are situated in the left-half complex plane 
and where G satisfies the equation: 



F=A-GC . 

x(t) is called estimate of x(t) . 

x(t) = x(t) — x(t) is the estimation error, and x(t)- 



(2.4) 



>0. 



Substituting (2.4) in (2.3), and remembering that y = Cx , one obtains 



x = Ax + Bu + GCx 



(2.5) 



Block Diagram. Equation (2.5), which is equivalent to (2.3), leads to the state- 
representation block diagram of Fig. 2.2, where both the system to control and its 
observer are represented, and where has been introduced the quantity y = C x . 



Plant 




H±y 



Observer 






H±y 



Fig. 2.2 Block diagram of a plant and its observer. 

Physical Interpretation. Equation (2.5) suggests the following remarkable inter- 
pretation of a Luenberger observer, illustrated in Fig. 2.2: The observer can be 
seen as a model of the plant, which is driven, not only by the control signal u of 
the latter, but additionally by the estimation error x , in fact by Cx = y , weighted 
with a certain factor G. 
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Let us comment this. The Luenberger observer can be considered as being built 
in two steps: 

1°) a simulation of the plant is realized by means of a 

model, which yields a reconstruction x of x as 
sketched on the right, but which, used like that, has 
the following drawbacks: 

• if the model does not match perfectly the plant, 
x(?) will not tend exactly towards \(t) when t — > oo ; 

• if one applies a state feedback from x , namely u = — Lx , the resulting control 

is finally an open-loop control, and not a feedback control system, since the 
measurement (output vector y) is not used; 

• if the plant is unstable in open loop, its simulation model will not deliver any 
usable signal; 

2°) a correction term is thus added, containing y, or more precisely proportional to 
the estimation error, Cx = Cx Cx = y — y = y , i.e. to the difference between 

real measured output y and estimated output y . 



2.2.1.2 Discrete Case 



The reasoning is very similar to the continuous case. Let a discrete plant be de- 
scribed by: 



We will try to build another linear system, with state equation 



x i+ i=Fx i +Ju i +Gy i , 
such that the estimation error 



(2.6) 



X z. — X . X . 



tends towards zero when t — ► oo . 

We write for that purpose the difference state equation which describes the 
evolution of the estimation error as a function of time: 



v *+l — A /c+l A A-+1 



x i+1 =*x t +Tu k -Fx t - Juj -Gy t , 
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thus, with y t =Cx t , 

x t+1 =Fi t +(*-F-GC)x t + (r-J)u t . (2.7) 

We wish here again to cancel as many terms as possible in the second member of 
this equation. By letting J = T , and imposing to F and G to satisfy the relation 

O-F GC = 0, (2.8) 

equation (2.7) becomes 

X k+l =F X £ • 

This homogeneous difference equation represents the free response of a dis- 
crete-time system with system matrix F, and has, according to (A. 19) of Appendix 
A, the solution 

x t = F*x(0). 

If the eigenvalues of F are within the unit circle, x, — >0 . 

Equation (2.6) in turn becomes 

x t+1 =Fx,+ru t +Gy, . (2.9) 

By substituting, according to (2.8), 

F = 0-GC , (2.10) 

(2.6) can also be put in the form 

x k+l =Ox k +ru k +GCx k . (2.11) 

The similarity with the continuous case formulae is obvious. 

The block diagram is thus the same as the one of the continuous case, where A 
is replaced by O , B by T and the integrator block by a one sampling-period de- 
lay element. 

2.2.2 Synthesis of the nth-order Observer 

The design occurs in two steps: 
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1. the eigenvalues of the F matrix are chosen, in the left-half complex plane in the 
continuous case, respectively inside the unit circle in the discrete case, gener- 
ally to the left of those which have been prescribed to the closed-loop system, 
i.e. those of A — BL , respectively closer to the center of the unit circle than 
those prescribed for O — TL , so that the transient response of the observer de- 
cays faster than that of the closed-loop system; these eigenvalues should how- 
ever not be chosen too fast, the observer having then the tendency to amplify 
high frequency noise, as a result of an excessive bandwidth; 

2. the matrix G is determined so that F = A GC, respectively F = 0-GC, 

has indeed these eigenvalues. Let f(s) , respectively /(z) , denote the corre- 
sponding characteristic polynomial, i.e. the polynomial whose roots are these 
eigenvalues: 

f( S ) = S n +f n _ 1 s"- 1 + -. +/,* + /„, 
or /(z) = z"+/„_ 1 z- 1 + - +/,z + / . 

To simplify, we will continue the reasoning in the continuous case only. We 
must thus find a matrix G such that 

dct(sl-F) = f(s), 
thus 

det[*I-(A-GC)] = /(s). (2.12) 

Since det M = det M T holds for any matrix M, (2. 12) is equivalent to 

det [si -(A T -C T G T )] = /(*). (2.13) 

Formulated this way, the problem is quite similar to an already discussed one, 
the problem of a state-feedback design by pole placement. It is enough, indeed, to 
apply the following correspondence: 

State-feedback Synthesis: Observer Synthesis: 



A ► A T 

B ► C T 

L > G T 



(2.14) 



to recognize that (2. 13) is formally identical to 
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detfsI-(A-BL)] = /?(*). 
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Remark further that (2.14) induces as a corollary the following correspondence: 



B AB A""B 



-» (C T A T C T 



(aT) „-i c t 



which can be written, since (M 1 )* = (M*) T for any matrix M, 



C 
CA 



B AB A" _1 B 



This spells literally as: 



controllability 
matrix Q c 



CA" 



observability 
matrix Q 



(2.15) 



It had been shown in Chap. 1 that the pole placement problem has always a so- 
lution if the plant is controllable. From the last correspondence we deduce there- 
fore that the observer synthesis problem is solvable if the system 
{x = Ax + Bu, y = Cx} is observable. The observer synthesis is a dual problem 

of the controller synthesis in the duality exposed above, due to Kalman [Kal60]. 
For a mathematical or a practical observability criterion, see Appendix A. 



2.2.3 Observer Determination for Single-output Systems 

This duality shows that the design problem considered here has a unique solution 
if q = 1 , thus if the system has only one output: 



x = Ax + Bu 

[y = c J x 

The observer representation becomes, instead of (2.3) and (2.4): 

x = Fx + Bu + g>>, where F = A gc . 



(2.16) 



The present problem is dual to the design of a state-feedback control for a single- 
input system. By adapting the previous duality to this scalar situation and complet- 
ing the correspondence (2.15) by 
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last row of Q c 



T 



last row of (qJ I = last column of Q^ 1 , 



T 

q > 



(2.17) 



it will be possible to transpose directly by duality the results of Chap. 1. 
2.2.3.1 Plants in Observability Canonical Form 

Theorem 2.1. If one assumes that the plant 

x = Ax + Bu 

T 

y = c x 

is given in observability canonical form, with the characteristic polynomial 

s"+a n _ l s"~ l + ■■■ +a { s + a , 



and if one prescribes for the Luenberger observer eigenvalues of F = A — g c 
such that the characteristic polynomial is 

S" + /„_! S"' 1 + ■■■ +fs + f , 

the vector g must take the following value: 



g = 



Jo ~ a o 

Jn-\ ~ a n-\, 



(2.18 



Let us write in this case the observer equations explicitly. Since the plant to 
control is then represented by the following matrices (see relations (A.3) or (A.34) 
of Appendix A): 



— a 



(o - i -«„-J 



(ho 1 



\ u n-\ 



(0 ••• 1), 



we have 



2.2 Full-state, or nth-order (Luenberger) Observer 



87 



F = A gc 1 =A 



••• 1 = 



••• 

1 ••. 



- a o-8i 



••• 1 -a. 



and, with (2.18), 

-/o ^ 

i '•. : 

F= . . . . 

- 1 -fn-l, 

The observer itself is thus represented in observability canonical form. 



(2.19) 



2.2.3.2 Plants in Arbitrary State Representation: Use of 
Ackermann's Formula 

By means of the correspondences (2.14) and (2.17) we deduce from the relation 
(1.24) the following dual formula: 



g T = / qJ+/iq o T A T 



-f n _,q^(AT l + q ^A T r, 



which leads, after transposition, to the statement of a dual theorem of Theorem 1.2 
(Ackermann's theorem): 

Theorem 2.2. If the plant x = Ax + Bu is observable, and if one desires that the 
nth-order (Luenberger) observer has the characteristic polynomial 



f(s) = s"+f„_ l s"- l + -. +f lS + f , 



one must choose 



g = /oq +/; A qo- 

- /(A) q a 



-f„_ l A"- 1 q +A"q 



(2.20) 



where q is the last column of the observability matrix inverse, Q^ , and is de- 
termined by 
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c T q = 








'0 


c T A"" 2 q = 


or 


q : 


= Qo 





c T A" _1 q =l 








y l 



2.2.3.3 Example 



Consider the plant represented on 
the right. Build a full-state observer 
for this plant, having a double ei- 
genvalue s = — 3 . 



u 


' i ^ Xl 




y * + 1 1 




s + 2 



x 2 = y 



The plant state representation is established by mere inspection: 





X = 

y = 


Ax + bi* 
c T x 




A = 


'-1 
1 


0' 
-2, 


j 


b = 


1' 




Q 


= 


,c T A ; 


= 



1 


1 % 

-2 ; 


=> 


Qo 


= 


2 
1 


f 

0, 



c'=0 1 



(s + 3) 2 =/(s) = /+6s + 9 => f 1= 6,f =9, 



Aq c 



-1) 
1 



, A q =A Aq c 



n 

3 



Applying (2.20) yields 



g = 9 



'r 


+ 6 


-r 


+ 


' r 




V 


k 




[ i, 




(-3, 




[i J 



then, with (2.16), 



F = A-gc 1 



[-1 


o x 




^ 


(0 1) 


f-1 


-4 s 


1 


-2 




V 3 , 




1 


-5 



The resulting observer has finally the following equation: 



-1 


-4" 


x + 


V 


u + 


'4* 


( 1 


-5 




(o, 




y i) 
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2.3 Reduced-order Observer 



2.3.1 Basic Idea 

The «th-order observer studied previously, and aimed at observing a system which 
is also of «th order, has some degree of redundancy. This results from the fact that 
this observer builds an estimate of the entire state vector, although part of it is al- 
ready accessible to the measurement, according to the plant outputs. 
Consider again a continuous-time plant described by 



x = Ax + Bu 

y = cx 



The q outputs being linearly independent, which means to say that the output ma- 
trix C, of size (qxn) , has rank q, it is always possible to assume, eventually with 
a change of basis, that this matrix takes the form 



••• j 1 


... o 


••• | 


... i 


n — q 


q 



("I 1 , 



(2.21) 



Indeed, if initially the system is represented differently and has the output equa- 
tion y = C'x' , the change of coordinates 



Tx' 



where the matrix D, of size (« — q) x n , is chosen such that T is regular, will lead 
to the above representation. 

2.3.1.1 Partitioning 

Let us then rewrite the system state equations, by partitioning them while taking 
into account the new form of C, thus by writing 
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X, 






X n-q 




{ \ 

V 


X n-q+\ 




^ y ; 


X n 







J n-q 



(2.22) 





f • ^ 
V 




(A A ) 


V 




f B 


X = 





•^z. 


1 


— 


+ 







y t 




A 21 j A 22 ^ 


y, 




B 2 



} n-q 

::.} q ■ 



or 



n — q q 

v = A 11 v + A 12 y + B 1 u, 

y - A 22 y - B 2 u = A 2i v • 



(2.23) 

(2.24) 
(2.25) 



It will thus suffice to estimate part v of the state vector x, i.e. (n — q) components. 



2.3.1.2 Idea of the Reconstruction 

Since y is accessible to the measurement, y is it also. Since u is also measurable, 

equations (2.24) and (2.25) can be considered respectively as the state differential 
equation and the measurement equation of a system which would have, 

• as state vector, v, 

• as input vector, A 12 y + Bj u , 

• and as output (measurement) vector, y — A 22 y — B 2 u . 

A full-state Luenberger observer, thus of order (n — q), is then built for this sys- 
tem. According to (2.3), if we introduce instead of G a new gain matrix G r of 
size (n — q) x q , it will have the following differential state equation: 

? = (A 11 -G r A 21 )v + A 12 y + B 1 u + G r (y-A 22 y-B 2 u). 

A priori this observer does not appear satisfactory, since it involves the derivative 
of y. To avoid this inconvenient, let us define 



z = v-G r y. 



(2.26) 
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2.3.1.3 State Equations of the Reduced-order Observer 

Substituting v from (2.26) in the previous equation, we obtain: 

z = (A n -G r A 21 )z + [(A n -G r A 21 )G r +A 12 -G r A 22 ]y+(B 1 -G r B 2 )u 



v = z + G r y 
These equations lead to the block diagram in Fig. 2.3: 



(2.27) 




Fig. 2.3 Block diagram of a reduced-order observer. 

Since v , thus also z, have the dimension (n — q) , this is well a reduced-order ob- 
server, of order (n — q) . 

Lemma. If the pair (A, C) is observable, the pair (A n ,A 21 ) is it also. This prop- 
erty stems directly from the definition of observability. 

Theorem 2.3. Given a linear, time-invariant system of order n, which has q line- 
arly independent outputs and is assumed observable, it is possible to construct an 
observer of order (n — q) having arbitrary eigenvalues. 

Remark 2.1. The previous construction yields one observer of this type, which has 

F = A 11 -G r A 21 . (2.28) 

as system matrix. There are other solutions, as e.g. in the following situation. 

Case where C =(I ? j 0) in (2.21). The solution is then obtained by simple per- 
mutation of the indices 1 and 2 in (2.27) and (2.28). The corresponding system 
matrix, for instance, becomes then 



F = A 22 -G r A 12 . 
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Theorem 2.3 claims that it is always possible to reach the fixed objective, pro- 
vided the hypotheses of the theorem are fulfilled, and this, by imposing arbitrarily 
the (n — q) eigenvalues of the corresponding F matrix. 



2.3.2 Reduced-order Observer Design for Single-output 
Systems 

Consider again a plant of the form 



x = Ax + B u 

T 

y = c x 



Now, partition the matrices and vectors as previously, but taking into account 
the fact that q = 1 . The partition for x becomes here, according to (2.22): 



} n-\ 



It ensues, according to (2.23), that A and B are partitioned as follows: 



A„ 


a 12 


a T 

21 


! «22 



M-l 



B 



}1 



B 



n-1 



}1 



n-\ 1 



and that c T must have, according to (2.21), the form c T = (0 



(2.29) 



1) , or 



should be put in it. 

The state equations of the reduced-order observer take then, instead of (2.27), the 

following form, with G r = g r : 



Z = ( A ll-gr a 2l) Z + [( A n-g,- a 2l)g r + a 12-g r «22]>' + ( B l-g,bJ)u 

from what results that: 



(2.30) 
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F = A n -g r a^ . 
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The design problem consists thus in prescribing the (« — 1) eigenvalues of F, 
which amounts to imposing its characteristic polynomial 



f(3) = S n - l +f n _ 2S "- I + ■•■+/! S + f 



(2.32) 



and determining g r such that F has effectively this characteristic polynomial. 

The above equations can be solved directly by polynomial coefficients identifi- 
cation in the case n is not too large. If this is not the case, one of the following de- 
sign methods can be used. 



2.3.2.1 Case of a SISO Plant in Observability Canonical Form 

In this case: 



fO ••• -a ) 



[0 - 1 -a n _y) 



(bo ) 



\ u n-\) 



c 1 = (0 ••• 1. 



Since c T has the form (0 j \ q ), no change of coordinates is required. Thus, ac- 
cording to (2.29): 



a*=(0 



••• 0) 



1 



l n-2) 



B, =b' = 



\K-2, 



1), a 22 = -a n _ x , bJ=Z>„_! 



The posed problem consists then in finding g r = lg r { ■■■ g r n _ x \ such that F 

given by (2.31) has the characteristic polynomial given by (2.32). If one recog- 
nizes the formal similarity of this problem with the one solved by Theorem 2.1, 
where one would have reduced n by one and made a = a i =--- = a n _ { = , the 
solution is obtained readily, according to (2.18): 



8r=(/o •'• fn- 
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(0 - -/«, 
1 '•. : ; 

• ••. o : 

,0 - i -/,_ 2 J 
by resemblance with (2.19), and finally, by application of (2.30): 

z = Fz + [Fg r + a n +a n _ l g r }y + (b'-b n _ l g r )u 
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(2.33) 



2.3.2.2 Case of an Arbitrary State Representation of the Plant 

We apply then the dual Ackermann's formula, Theorem 2.2, formula (2.20), to the 
reduced system to be observed, of order (n — q), given by the equations (2.24) 

and (2.25), thus with system matrix A = A u and output matrix c T = a 21 . The re- 
sulting gain vector g r is then inserted into (2.30) and (2.31). 



2.3.2.3 Example 

Consider again the example solved in Sect. 2.2.3.3. The plant being a second order 
system with one output, it should be possible to build a first order observer having 
an arbitrary eigenvalue. Let us determine this observer for an eigenvalue s = — 3 . 

The plant matrix c T = (0 1) has already the required form. The plant is not 
here in observability canonical form. We will thus apply successively the direct 
calculation and the method indicated above (dual Ackerman's formula). 

The partitioning according to (2.29) yields here: 



A = 



l"21 



i 22 



a n = — 1 , a n = 

CI-) I — i « CI-)-) — z. 



; b 



[b 2 



\ = \, b 2 =0. 



Here, g r becomes a simple scalar g r . Applying (2.31) we can thus write: 



V = a n -g r a 2l = -l-i 



det(sI-F) = s + l + g r =f(s) = s + 3. 
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Therefore: 



Moreover: 
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g r =2, 

F = -l-2 = -3. 



b l -g r b 2 =l-0 = l, 

a l2 -gr a 22= Q + 2 gr= 4 - 

Finally, the reduced-order observer has, according to (2.30), the expression: 

\z = — 3z — 2y + u 
[%=z + 2y 

These equations lead to the block diagram of the observer in Fig. 2.4, con- 
nected to the plant: 




Fig. 2.4 Block diagram of a first-order observer for a second-order plant. 

The dual Ackermann's formula applies to the reduced system, thus here to the 
pair (c T , A) , with c T =a 2l =l and A = a u = — 1 . We obtain thus, successively, 



Qo=c =1 =*> q =l, 



and, with f(s) = s + 3 , 



g r =fo<lo+fi A< lo= 3 - ] - = 2 - 



Finally, with (2.31) and (2.33), F = A-g,.c' =-1-2 = -3 and 



\z = —3z — 2y + u 
v = X\ = z + 2y 
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This result is obviously identical to the previous one. 

2.4 Generalized Observer 



The previous concepts of «th-order observer and reduced-order observer can be 
grouped under one single theory, thanks to the generalization which will be pre- 
sented hereafter, and which will additionally allow introducing a new form of re- 
constructor: the functional observer. 



2.4.1 Generalized Observer Definition 

To observe a linear time -invariant system, assumed observable, of order n and 
state vector x, another linear time-invariant system can be introduced, of order r 
and state vector z, and having the following state equation [CeBa84] : 

z = Fz + Ju + Gy, where size(z) = (r x 1) . (2.34) 

Such a system is said to be an observer of the first one if, given an arbitrary trans- 
formation matrix T, of size (r x n) , its state vector z represents an estimation of 
Tx, vector of length r: 

z = Tx , (2.35) 

z Tx. (2.36) 



i.e. 



Introduce the estimation error 

£ = Tx z = Tx Tx. (2.37) 

It is thus desired that £ >0 . 

Let us establish the state equations of this generalized observer by following 
the same reasoning as has been done for the «th-order observer. Since the estima- 
tion error £ must tend towards zero for t — * °° , it must be possible to write: 

£ = F£, (2.38) 

or 
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(z-Tx)*=F(z-Tx), 

where F is a matrix of size (rxr) , the eigenvalues of which will be situated in the 
left-half complex plane in all cases. 
It comes out that 

z = Fz + Tx FTx, 

and, since x = Ax + Bu , 

z = Fz + (TA FT)x + TBu . 

The fact that x is accessible only in the form y = C x suggests to look for a matrix 
G such that 

(TA FT)x = Gy, 

thus such that 

TA-FT = GC , (2.39) 

where G is of size (rxq) since C has size (qxn) and since F and T have the 

above mentioned sizes. 

The state equation (2.34) becomes then, with J = TB , 

z = Fz + TBu + Gy . (2.40) 

The observers of order n and of reduced order are deduced from this general 
representation as special cases. 

2.4.2 Special Case of the Reduced-order Observer 

We dispose here of q measurements and want to construct an observer of order 
r = n — q which reconstructs the (n — q) non measurable components of x. 

With the previous hypothesis that C = ( j I J , the vector to estimate is (see 

(2.22)) 

_ v } n — q 

x= — 

y \q 
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By rewriting (2.26) the following way: 



z = v-G r y = I ! -G, 



z will have indeed the form (2.35), z = Tx , provided we let 



T = (I„_ ? ! -G, 



(2.41) 



Let us verify whether, with this transformation T, the relations (2.27), which had 
been obtained with the form of C recalled above, are effectively regained. 
F will be here a matrix of size (n — q) x (n — q) , and G will be of size (n — q)xq. 
By applying (2.39), we obtain, with the partition defined in (2.23) for A and B, 



V, ! "G, 



A u | _A_i2 
A 2i ! A„ 



F(I„-, !-G, 



G(0jl, 



'r-^21 I -^12 



from where it follows readily that 



F=A n -G r A 21 , 

G = FG,. + A,,-G 1 .A„. 



B being also partitioned as in (2.23), we have also 

B[-G r B 2 



TB = [l n _ q j-G, 



B 
B, 



(2.42) 



By substituting the values of F, G and TB into (2.40), we regain equation (2.27). 
Remark 2.2. The reduced-order observer corresponding to the situation where 



C = (I j 0) is obtained by letting 



G, |I 



I »- 



(2.43) 
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2.4.3 Special Case of the nth-order Observer 

This case amounts to considering that no measurement at all is available, thus set- 
ting q = and constructing an observer of order r = n . 

It seems then that we could limit the second member of (2.40) to its first two 
terms. The fact that we want however to conserve the term in y confirms the re- 
dundant character of such an observer. 

The matrix T is here of size (n x n) . The estimated vector z has, in this case, 

the right dimension n. If one wants to have z — x , it is necessary to let 

T = I„ = I . (2.44) 

This is the reason why the previously described «th-order observer is also called 
identity observer. We regain here again the relations of this observer; in particular 
(2.39) becomes A — F = GC , which is nothing else than (2.4). 



2.4.4 Generalized Observer Output Equation 

In the previous sections it was considered that the state vector z of the general- 
ized observer was giving directly the desired estimated quantity. This is why no 
output equation was needed. 

It may be however interesting to enlarge the representation of the generalized 
observer given by the state equation (2.40), by adjoining to it an output equation 
and thus a generalized output vector T| , of an arbitrary dimension m < n , as in 

[LaTh77], [CeBa84]: 

Tl = Hz + Ky , (2.45) 

where size(H) = (m x r) and size(K) = (mxq) . 
By using (2.36), the following will thus hold: 



limTl = Hlimz + Ky = HTx + KCx = (HT + KC)x= H K 



x (2.46) 



In the case where the generalized observer output vector must tend, for t — > °o s 
towards the state of the observed system, which is the case of the two types of ob- 
servers, or state reconstructors, seen previously, it is of course possible to include 
them into this more general representation, provided we make there m = n and 
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H'K 



T 
~C 
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(2.47) 



which will have the consequence that 



lim T) = x . 



2.4.4.1 Application to the Reduced-order Observer, Case where 



c = (°!^ 



With the use of the expression (2.41) of the matrix T which corresponds to this 
case and with the help of (B.4) of Appendix B, we can rewrite (2.47) as follows, 
since the matrices \ n _ q and \ q are obviously invertible: 



H |K 



K-q 1 G r 


-1 


K-q 1 G r 


o--|-f- 




o"tf~ 



or, finally: 



H 



and K 



I 



(2.48) 



2.4.4.2 Application to the Reduced-order Observer, Case where 



C = ( I ? !° 



By substituting this time (2.43) into (2.47) and using (B.5) of Appendix B, one ob- 
tains 



H |K 



T' 


-l 


f r li ~> 
"r 1 l n-q 


-i 





h\ 


"c 


n—q 


G r 



or, finally: 



H 



and K 



i 

g" 



(2.49) 
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2.4.4.3 Application to the Identity Observer 

With here T = I n according to (2.44), the only way to satisfy (2.47) V C is to let 

H = I„ and K = . (2.50) 

2.4.5 Application: Functional Observer 



2.4.5.1 Principle 

For some applications, an estimate of a simple linear combination of the state vari- 
ables £ = 0t T x is all what is needed. It is the case, e.g., for a linear control law of 
a single-input system, u = —t x . 

The following question arises thus: is it possible in such situations to build a 
less complex observer than if the entire state were to be reconstructed? 

We are faced here with the particular case of the output equation (2.45) corre- 
sponding to m = 1 , i.e.: 

£ = h T z + k T y , 

where z is the observer state vector and y the plant output vector. 

For t — > oo , £ must tend towards £ = 0C T x . Since then z — > Tx , (2.46) yields 

cc T x = (h T T + k T C)x, 
and h T T + k T C = cc T . (2.51) 

The row vectors h T , k T and 0C T are respectively of dimensions r, q and n, if one 
assumes that the observer order is r (see Sect. 2.4. 1). We are thus faced with a sys- 
tem of n equations to determine the r + q unknowns of h T and k T . 

A priori, we could be tempted to choose r + q = n , thus to design an observer 
of reduced order r = n — q to estimate £ . The particular interest of the linear 
combination estimator resides in the following result, established by [Lue71]: 

Proposition 2.1. An arbitrary linear combination of the state, namely £ = a T \ , 
can be estimated by means of an observer having (v — 1) arbitrary eigenvalues, 

thus of order {y — 1) , where v is the observability index defined as the smallest 

positive integer for which the following matrix has the rank n: 



102 



2 Observer Theory 



c 

CA 

CA ;/ 



Since, for any completely observable system, v — 1 < n — q and since, for many 
systems, v — 1 <C n — q , the observation of a linear combination can be much sim- 
pler than that of the state vector itself. 

The synthesis takes place as follows: 

1 . a matrix F of size [y — 1) x (v — 1) is chosen, with imposed eigenvalues; 

2. ot T being imposed (in the case of the state-feedback control of a single-input 
system, ot T = — £ T ), as many parameters of h T , k T and T as possible are de- 
termined, such that (2.51), namely h T T + k T C = 0C T , is satisfied; 

3. T and G are determined such that the relation TA FT = GC is satisfied. 

The observer obtained this way is called control observer or functional ob- 
server, because it estimates a function of the state variables and not these variables 
themselves. It is then given by the equations: 



z = Fz + TBu + Gy 
£ = h T z + k T y 



(2.52) 



2.4.5.2 Example 

Consider the 4th order system represented in Fig. 2.5 '. 





5 + 1 




s + 2 




5 + 2 



Fig. 2.5 Fourth-order system used in the example. 

The available measurements for this system being x l and x 3 , let us show that it is 
possible to estimate any linear combination of its state variables by means of a 



This example is taken from the paper [Lue71]. 
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functional observer of first order. We will then construct such an observer having 
a single eigenvalue s = — 3 , which estimates the linear combination x 2 +x 4 . 

The inspection of the functional relations between the four variables, relations 
all of first order or of pure integrator type, yields directly the following state rep- 
resentation: 



x = 



'-2 


1 





o N 




'0 s 






-2 



1 
-1 
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x + 
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Observability index calculation. Build the matrix composed of the first 2 rows 
of the observability matrix: 







' 1 















'c x 
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-2 
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1 








=*> det 


c 

CA 












-1 


1 







= 1^0 => v = 2 



The observability index of this system being equal to 2, it will thus be possible to 
calculate a functional observer of first order which estimates a linear combination 
of the state variables. 

Functional observer calculation. From (2.52) we have 



i = Fz + TB M + Gy 
[£ = h T z + k T y = a T x = x 2 + x 4 



size(F) = O - 1) x [y - 1) = (1 x 1) 
size(T) = (lx4) 
size(G) = (lx2) 
size(oc T ) = (lx4) 



Let therefore 

F = /;T = (f, t 2 t 3 t 4 );G = { gl g 2 ); a J = (0 1 l) . 

The observer characteristic polynomial is 

det(sI-F) = (s-f) = s + 3 => / = -3. 
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The equation 

h T T + k T C = a T 

becomes, with the present dimensions, 

hT + (k l k 2 )C = a J , 
or: 

h{t { t 2 t 3 u) + (k x k 2 0) = (0 1 1) 
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(2.53) 



If we take h — 1 , so as to have for the estimated combination the equivalent of 
v = z + Ly (see (2.26)), solving (2.53) yields 

t 2 =t 4 =l => T = (t l 1 t 3 l) and TB = t 4 = 1, 

k\ = ~~ 1\ 7 
k 2 = —t 3 . 

It remains now to determine t x , t 3 , g } and g 2 in such a manner that the equality 
TA FT = GC is satisfied. With GC = (g, g 2 0) , this condition writes 



'i 1 h 1 



•3(f, 1 h l) = (ft g 2 0), 



(-2 1 0) 
0-2 10 

0-11 

1 

-2^-1 + %= gl , 

^-2+3=0 =*> ?i=-l => g\ =-2 and ^ =1; 
l-? 3 + 3? 3 =g 2 , 

? 3 + 3 = =4> ? 3 =-3 => g 2 =-5 and £ 2 =3. 



The obtained functional observer can be represented by the block diagram of 
Fig. 2.6. It is as expected a first order dynamic system. A reduced order observer, 
which would have estimated separately x 2 and x 4 , would have been of order 2. 
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Fig. 2.6 Functional observer estimating a linear combination of 2 states. 



2.4.6 Generalized Observer Design Method 



A general method to solve (2.39) has been presented in [CeBa84], by using a pre- 
vious result of [B0Y068]. Due to the complexity of the calculations involved, only 
the resulting theorem established in this work will be given here without proof. 

Theorem 2.4. If the pair (C, A) is observable and if the spectrum of the matrix F 

(i.e. the set of its eigenvalues) is distinct from that of A, the equation 
T A FT = GC has one unique solution, given by the following formula: 



T = G,FG,...,F r_1 G 



{Cq,(A)) 
Cq 2 {A) 

lC<? r (A)J 



q ( A ), 



(2.54) 



where the polynomials q t (X) are given by: 

q (X) =/T +a,._ 1 /T~ 1 +... + a 1 A + a =(A-A Ll )(A-A L2 )...(A-A Ll .) 
q x (/i) = X"~ +a r _ x X~ +... + a 2 A + a l 



q t (X) =A'''+a r _ l A r ' +...+a ; 
qM) =1 



(2.55) 



According to [CeBa84] again, the synthesis approach of an observer of order r 
in its generalized form (2.40), be it for a full-state (r = ri) or a reduced order 
(r = n — q) observer, is then the following: 
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1. choose an (rxr) matrix F in one of the canonical forms, e.g. the controllabil- 
ity canonical form, which has the r eigenvalues desired for the observer, 

\n ' ■ • • ' "\> ' 

2. choose randomly a matrix G (rxq) , and deduce from it the matrix T (rxn) 
which is solution of (2.54); 

(T) 

3. if rank = r + q , continue, else go back to step 2, and make there a different 

choice for G; 

4. the desired observer is then given by (2.40) and (2.45) or one of its variants. 

Remark 2.3. In this design, the matrix G plays the role of the parameter matrix P 
of the complete modal control (Sect. 1.6). 

Remark 2.4. Let us recall here that it is the above theorem which has lead to the 
establishment by duality of the general design formula of a MIMO state feedback 
of Theorem 1.7 (Sect. 1.7) [BeOs87]. 

Remark 2.5. It is clear that the previous approach can also apply to the design of a 
functional observer. The proof is given in the above cited publication. 



2.4. 7 Other Design Methods of MIMO Observers 



2.4.7.1 Pole Placement Methods and Modal Methods 

The Kalman duality highlighted in Sect. 2.2.2 allows obviously using almost all of 
the state-feedback control design methods seen in Chap. 1 for the design of ob- 
servers, in the continuous as well as in the discrete case. Let us cite in particular 
the eigenvalue placement methods for MIMO systems, such as the complete mo- 
dal synthesis of Sect. 1.6. They allow by duality to select for all the eigenvalues of 
the observer arbitrary values if and only if the plant to be observed is observable. 

In the discrete case, it is of course also possible to synthesize a deadbeat ob- 
server [AsWi97], which has thus all its eigenvalues in the origin of the z-plane, by 
duality with the design of a control law of this type described in Sect. 1.3. The re- 
sulting observer is the fastest possible one, converging to the state of an «th-order 
discrete system in at most n time steps. 

It is enough, indeed, to replace in all those control design algorithms the matri- 
ces A and B respectively by A T and C T , the control matrix L obtained in return 
yielding in this case the transpose of the gain matrix G of the full-sate observer or 
G r of the reduced-order observer. This will be illustrated in the solved exercises 
proposed at the end of this chapter. 
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2.4.7.2 Application of the Complete Modal Design: Eigenvector 
Choice 

Let us transpose briefly the method described in Sect. 1.6 and consider for simpli- 
fication purposes the continuous time «th-order observer. The discrete case is 
quite similar. 

Once the n eigenvalues of the observer system matrix F = A — GC , solutions 
of the equation 

det[A ffi I-(A-GC)] = 0, i = \,...,n, 

have been chosen, the additional degrees of freedom will be used, by introducing 
this time the left eigenvectors w Cl of F, or right eigenvectors of F T , which sat- 
isfy the equation 

w c ,[(A-GC)-A G! .l] = 0, / = 1,...,«. 

The introduction of n parameter row-vectors q^ = w ffi G , of dimension q, 
changes the previous equation to 

w G! .(A-A G ,I) = q[C, 

which writes also 



T T 

w ffi q, 



fA-Ae,!) 



= . (2.56) 



Equation (2.56), dual of (1.67), represents a system of n linear equations in 
(n + q) unknowns. If the A Gl are all different from the eigenvalues of A, the col- 
umn matrix in the first member is of full rank equal to n, and a unique solution of 
this equation will be obtained if one chooses arbitrarily the value of q unknowns. 
The arbitrary choice of these q unknowns can be spread arbitrarily over the com- 
ponents of w Gi and those of q ; . It is guided by the following considerations. 

The observer transient response, solution of its state equation (2.3) expressed as 
a function of its eigenmodes, is obtained by replacing, in the formula (A. 45) of 

Appendix A, \(t) by \(t) and A, by A Gi , and by taking into account the double 

input of an observer: 

x(0 = £ v Cl . e Xa V G ,x + £ y Gi f'e Xci ( '- T) w T c , [Bu(r) + Gy(r)] dr . 

.■I ;_1 " 
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In the light of the discussion at the end of this appendix, it appears that the 
choice of the left eigenvector w Gl influences the manner the mode e Gi is ex- 
cited by the various components of x , also by the various components of the 
control vector u(7) , through the components of w G; B , and finally also by the 
various components of the measurement vector y(t) , through the components of 

An interesting application of these properties may consist in avoiding that a 
particularly noisy measurement excites the fastest observer dynamics, by imposing 
to the corresponding component of the parameter vector q ; to vanish [DucOl]. 



2.4.7.3 Quadratic Optimization Methods 

This transposition by duality can also be applied to the design of a state feedback 
by quadratic methods, the so-called optimal control, which will be discussed in the 
next chapter, with the aim of designing an optimal observer. Since however the 
real interest of using quadratic optimization methods resides in the possibility to 
extend their application to the case of noisy systems, the synthesis of such observ- 
ers, which will become in fad filters in this stochastic context (see Sect. 2.1.2), 
will be presented in Chap. 4 with a completely independent approach. 



2.5 State-feedback Control System with Observer 

As already mentioned, the matrix L of a state-feedback u = L x for a control 

system is calculated first by assuming x accessible. During this synthesis it is of 
course paid attention to the stability and a satisfactory dynamic behavior of the 
closed loop. 

The following question arises then: if, as a result of a partially inaccessible x, 
an observer, i.e. an additional dynamic system, must be introduced in the loop, 
does the closed-loop system remain stable? 

It is important to know the answer to this question, furthermore to know how 
the introduction of the observer modifies the pole map of the closed-loop system. 



2.5.1 State Equations of the Closed-loop Including an 
Observer 

Consider again a linear time-invariant continuous-time plant: 



2.5 State-feedback Control System with Observer 
jx = Ax + Bu 

ly = cx 
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(A, B, C : constant) . 



Once equipped with a state feedback, this system will have the following control 
law, according to (1.2): 

u= Lx + My r . 

Since the input quantity available to the controller is not x, which is assumed inac- 
cessible, but only its estimate x delivered by the observer, the control law be- 
comes 



u= Lx + My r , 



(2.57) 



as illustrated in Fig. 2.7 in the case of the nth-order observer. 

As already known, the stability of a control system is an internal property and 
does not depend on the applied external quantities. To study it, it is therefore 
enough to consider the regulation behavior alone, thus letting y r = : 



Lx. 



(2.58) 



Plant 



i "T" J 




G> 



H±y 



Fig. 2.7 State-feedback control including an «-th order observer in the loop. 
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From the point of view of the observer, considered here in its generalized form 
described by (2.40) and (2.39), it is clear that u must depend on the plant output y 
and on the observer state vector z (see (2.45)): 

u = Hz + Ky, 

where H and K are constant matrices. 

With z = Tx , according to (2.35), and y = Cx , the two previous equations 
yield 

Lx = HTx + KCx. 

This expression must hold for all values of t, thus also for t — > oo , in which case 
x = x . Therefore 



L = HT + KC, 



(2.59) 



equation which generalizes (2.51) encountered in the case of the functional ob- 
server. 

By taking into account the state differential equation (2.40), the composite system 
{observer + controller} is described by the equations: 



z = Fz + TBu + Gy 
u = Hz + Ky, 



(2.60) 
(2.61) 



where F must be chosen such that its eigenvalues are stable. 

The block diagram of the control system is therefore as shown in Fig. !.■ 
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Fig. 2.8 Composite system {observer + controller} applied to a plant. 



2.5.2 Separation Theorem 



The composite dynamic system {plant + observer} is characterized by the com- 
posite state vector (x T z T 
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The study of the stability of this composite system will be simplified if one in- 
troduces, instead of z, the estimation error £ = T x — z already defined previously 

(see (2.37)). It is solution of the homogeneous differential equation (2.38), re- 
called here: 

£ = F£. (2.62) 

As to the plant state differential equation, it becomes according to (2.61): 

x = Ax + Bu = Ax + B (H z + K y ) 

= Ax + BHz + BKCx, 

and, with the introduction of (2.59): 



x = Ax BLx BHTx + BHz 
= (A BL)x-BH£ 



(2.63) 



The previous composite system is then described by the two state differential 
equations (2.62) and (2.63), i.e. by 



A-BL BH 
F 



(2.64) 



The characteristic equation of the system closed by means of the observer is thus 



AI„ (A BL) BH 
AT F 



= 0. 



or 



|AI„-(A-BL)| • |AI,-F| = 0. 



(2.65) 



Since the first of these two determinants yields the eigenvalues of the closed-loop 
system without observer and the second one the observer eigenvalues, the follow- 
ing theorem is obtained: 

Theorem 2.5, or Separation Theorem. The eigenvalues of a control system, 
closed by means of an observer and containing a plant and a controller both lin- 
ear and time invariant, are composed of the union of the eigenvalues of the 
closed-loop system closed without observer and of those of the observer. 
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2.5.3 Consequences for the Closed-loop System Stability 

A control system, containing an observer in the feedback loop and a linear, time 
invariant plant and controller, is stable, if the eigenvalues of the closed-loop sys- 
tem closed without observer and those of the observer lie in the left-half complex 
plane. 



2.5.4 State-space Design Steps 

As already seen in Sect. 1.2.2 for the single input systems, and as has been also 
proved otherwise for the multiple input systems, the eigenvalues of the closed- 
loop system without observer can be chosen arbitrarily, provided the plant is con- 
trollable [Won67]. 

Furthermore, as seen in Sect. 2.2 for single output systems and along the sub- 
sequent sections for multiple output systems, it is also possible to choose arbitrar- 
ily the observer eigenvalues if the plant is observable. 

Conclusion. If a linear time-invariant plant is (completely) controllable and ob- 
servable, it is possible to impose arbitrarily the eigenvalues of the closed-loop sys- 
tem including an observer and a linear time-invariant controller in the loop, as 
well as the eigenvalues of the observer. 

The consequence of Theorem 2.5 is thus, in other words, a complete separation 
of the two design problems, therefore the name given to it, and the following 
Separation Principle: 

1 . the controller (matrix L) is designed first, without any concern about the even- 
tual need of an observer; 

2. if then an observer must be introduced in the loop, this observer will not influ- 
ence at all the eigenvalues of the closed-loop system just designed in the previ- 
ous step; it will simply add its own set of eigenvalues to the other ones. 



2.5.5 Influence of the Observer Initial State on the 
System Response 

Let us discuss now the total response, i.e. the sum of free and forced responses, of 
a control system closed by means of an observer, in other words its response to a 
reference change and to non-zero initial conditions of the plant and of the ob- 
server. 

The plant is described, in the continuous case, by: 
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jx = Ax + Bu 

ly = cx 



(2.66) 



and the observer is represented by (2.5), reproduced here: 

i = Ax + Bu + GCx, 
x = (A GC)x + Bu + GCx. (2.67) 

The control law u is given by (2.57) (see also Fig. 2.7): 

u= Lx + My r . 

Let the estimation error appear explicitly in this equation: 

u = L(x x) Lx + My,. 

The Laplace transform of u(t) , V(s) = JL\u(t)] , is then: 

U(s) = L[x(s)-X(s)]-LX(s) + MY r (s), (2.68) 

where X(s) = C[x(t)] , X(s) = C\x(t)] and Y r (s) = C[y r (t)]. Apply now the 
Laplace transform also to the first equation of (2.66), with the use of (2.68): 

-sX(s) = AX(s) + BU(s) + x 

= (A-BL)X(^) + BL[x(j)-X(j)] + BMY r (i) + x . (2.69) 

Moreover, the subtraction of (2.67) from the first equation (2.66) yields 

x-x = (A-GC)(x-x), 
the Laplace transform of which amounts to 

s [X(s) - X(s)j = (A - GC) \x(s) - X(5)] + (x - x ) . 

Thus 

(sI-A+GC)\x(s)-X(s)j = x -x . 



114 2 Observer Theory 

Inserting this result into (2.69) one obtains finally: 

(5l-A+BL)X(5) = BL( 1 ?I-A+GCr 1 (x -x ) + BMY r (5) + x , 
thus: 

X(s) = (si - A + BL) ' BM Y r (s) + (si - A + BL) ' x + 

+(^I-A + BLr 1 BL( 1 ?I-A + GC)" 1 (x -x ) . (2.70) 

A close inspection of this formula is instructive in more than one aspect. Consider 
successively the three second member terms: 

• the first term represents the forced response of the closed-loop system to the 
excitation y r , with a dynamics dictated by the only eigenvalues of the closed- 
loop system, which would be closed directly by the state feedback L (eigenval- 
ues of the matrix A — BL ); moreover, the relation between Y r (s) and X(s) is 
exactly the same as if one would apply a direct state feedback, thus the matrix 
M calculated with this hypothesis suits also here; 

• the second term represents the free response of the closed-loop system to an 
initial plant state x , with a dynamics determined by the same eigenvalues as 
previously; 

• the third term reflects the free response of the closed-loop system to a differ- 
ence between the initial values of the plant and the observer, with a dynamics 
resulting from both the eigenvalues of the closed-loop which would have been 
controlled directly by the state feedback L and those of the observer (eigenval- 
ues of the matrix A — GC). 

This last point reminds of the separation theorem, but adds the following in- 
formation: if the observer starts from the same initial state as the plant, this third 
term disappears from the formula; in other words, the observer dynamics is then 
totally absent from the closed-loop response. 

Said differently again, if one could start the algorithm of the observer from the 

same initial state as the plant (x = x ) , the closed-loop response would be com- 
pletely independent from the choice of the poles of this observer, even if they 
were e.g. much slower than those chosen in the design of L. The value of x(t) 
would in a way be stuck to that of x(t) and evolve with it, in perfect synchronism. 

So, when does the observer dynamics make itself feel in the closed-loop dy- 
namic behavior? It does so in two circumstances. First, it is frequent that the plant 
initial state is not known and that one is obliged to start the observer from an arbi- 
trary initial state, different from that of the plant. The estimated state will then 
converge towards the real plant state with the observer dynamics only. Second, as 
soon as a disturbance is applied to the plant, which amounts to apply a new initial 
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state x * to it, the observer plays again its role by letting x(t) converge to- 
wards x(t) with the dynamics corresponding to its eigenvalues. 

This mathematical analysis is corroborated by the diagram of the closed-loop 
system including an observer (Fig. 2.7), where it appears clearly that, if at a given 

time y(t) = y(t) , the observer input through its gain matrix G is no longer fed by 

anything. From this time on, the observer behaves thus as a model of the plant, 
which will consequently evolve with the same dynamics as the plant, for as much 
as its model matches exactly the plant model. 



2.6 Deterministic Disturbances Compensation. 
Disturbance Observer 



In Sect. 1.8 it was shown that constant disturbances applied to the plant could be 
suppressed by including an integral action in the feedback loop. 

In the present section this subject will be extended to disturbances, which are 
not necessarily constant, but still of deterministic nature. The case of stochastic 
equation (plant) disturbances or measurement disturbances will be the subject of 
Chap. 4, devoted to optimal filtering. 

Due to the similarity of mathematical developments, only the continuous case 
will be discussed in details, the final results being then directly transposed to the 
discrete case. Consider the following continuous time system: 

x = Ax + Bu + Ev 

(2-71) 
y = Cx 

where v is a disturbance vector of dimension m influencing the state through a 
constant (nxm) matrix E, in accordance with the diagram of Sect. 1.8.1.4. 

Two situations can then occur, according to whether the disturbance is measur- 
able or not [F6190]. 



2.6.1 Case of Measurable Disturbances 

The state feedback control law u = — Lx being determined as seen in Chap. 1, 
it would be desirable, in order to compensate the effect of the disturbance, to add 
to it a term u v such that the following holds, according to [F6190]: 

Bu v +Ev = 0, (2.72) 
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thus such that u v = — B _1 Ev , which would imply of course that B be invertible, 
thus before all that p^n .In the infrequent case where this situation is realized, 
this compensation approach is nothing else than the MIMO extension of the an- 
ticipation compensation of the SISO case. The control law u = — Lx + u v yields 
then for the closed-loop system the equation 

x = Ax BLx BB ~Ev + Ev = (A BL)x , 

which confirms the full rejection of the disturbance. 

In the more usual case where p < n , (2.72) is overdetermined and it is possible 

to find only an approximate solution in the sense of the least squares, according to 
the well known method of Gauss. Since the second member cannot be equaled to 
zero anymore, one attempts merely to make 

£ = Bu+Ev 

as small as possible, in quadratic value: 

|e| 2 = e T e = (u^b t + v t e t )(Bu v + Ev) 

= u^B T Bu v + <B T Ev + v T E T Bu v + v T E T Ev 
= u^B T Bu v + 2u^B T Ev + v^E T Ev . 

According to the relations (B.10) and (B.8) of Appendix B, 

Ifil 2 = 2B T Bu v + 2B T Ev . 

<9u„ 



B being of full rank, equal to p, the matrix B T B of size (p x p) is regular, and the 

I |2 

minimum of £ in u„ will be obtained for 



u v = (B T B) 1 B T E v = BEv , 

where B^ = (B T B) _1 B T is the pseudoinverse or Moore -Penrose inverse of B. The 
control law is then 

u= Lx B j Ev or u= Lx BEv, 

the second equation corresponding to the case where an observer is needed to re- 
construct the state. It is this last case which is represented in Fig. 2.9, in which ap- 
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Fig. 2.9 Control system with observer and deterministic measurable load disturbance v. 

Since the disturbance is measurable, it is also injected into the state equation of the 
observer, which must have the same input signals as the plant. In the figure there 
is also a static gain compensation matrix M, as has been seen in Sect. 1.1. 



2. 6.2 Case of Non-measurable Disturbances 

Though the disturbance is not directly measurable, a knowledge of its form, at 
least partially, is often available, as is the case with piecewise constant distur- 
bances, the instants of the jumps and their amplitude remaining random, or of si- 
nusoidal signals. 



2.6.2.1 Disturbance Model 

The disturbance v is modeled as the output of a linear system, of state vector ^ 
and state representation: 



4 = A,$ 



v = C£ 



(2.73) 



with given initial conditions, [AsWi97]. The eigenvalues of A^ are commonly 

located at the origin of the complex plane or on the imaginary axis. 

Examples: 



• constant disturbance: A « = : 



undamped sinusoidal disturbance, of angular frequency ui Q : A 



c 



( u ) 
-w 
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2.6.2.2 Derivation of the Equations in the General Case 

This model is then added to the plant state model (2.71) and constitutes with it an 
augmented system, the state representation of which is given by the following 
equations: 



(2.74) 
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This state equation takes the form of a controllability canonical decomposition 
(see Appendix A, Sect. A.2.3). The state variables £ are not controllable, which is 

not a surprise: the disturbance can obviously not be influenced by the control vec- 
tor u. It is thus illusory to try and control this system by an augmented state feed- 
back, which would include the disturbance state vector, u = — Lx — L^ , even if 

the plant state x is entirely accessible. 

One turns then to the estimation of this disturbance by means of a disturbance 
observer, so as to be able to introduce an anticipation compensation based on this 
estimate, in a way similar to the approach followed in Sect. 2.6.1. 

Except in particular cases, the augmented state vector (x T \ ) is completely 

observable, if one assumes that so is the pair (C A). It is then possible to synthe- 
size an observer of the augmented system, which will deliver an estimate of both 
the plant state and the disturbance state, and then to choose a state-feedback con- 
trol law, which is a linear function of these estimates: 



u = Lx -L;^. 

The observer state equation is, according to (2.5): 



(2.75) 



' * x 
X 
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A f, 


I 




0, 




G f, 



C(x-x). 



(2.76) 



Substituting (2.75) into the first state equation of (2.74), to which the term — BLx 
is subtracted, then added, yields for the closed-loop state equation: 

x = (A BL)x + BL(x x) + EC£ - BL^f . 
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By introducing the estimation errors of the plant state, x = x — x , and of the dis- 
turbance model state, £ = £- ^ , the closed-loop equations become: 



x = (A - BL)x + (EC ? - BL f )£ + BLx + BL f % 

x = (A-LC)x + EC f | 
f = A # |-G f Cx 



(2.77) 



The state feedback matrix L is determined as previously, by one of the methods 
described in Chap. 1. A judicious choice of the gain matrix L« allows reducing 

the effect of the disturbance, in particular if it is possible to satisfy 
EC f -BL f =0. 

The block diagram of such a control system is shown in Fig. 2.10. 

x 



^(m}4 




Fig. 2.10 Control system with disturbance observer and a non-measurable load disturbance v. 



2.6.2.3 Particular Case of a Constant Disturbance, Acting at the 
Plant Input 

This case is interesting, since it corresponds to the dry friction of the actuator of a 
mechanical plant. Such a disturbance is modeled by (2.73) with A^ = and 

Ce = I , thus v = E, . This disturbance being applied to the plant control input, 

one has also E = B . The equations (2.77) are written here: 
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x = (A BL)x + B(I - L^)£ + BLx + BL f % 

x = (A LC)x + B| 
%= G^Cx 

The disturbance can be eliminated completely from the state equation by 
choosing L^ = I . The control law (2.75) and the observer state equation (2.76) 

are then given respectively by 

u = -Lx-L«^= Lx — v , 

x = Ax + B(v + u) + GCx 
v = f = G^Cx 



and 



Remarking that the second equation can also be written 

v = G^C fx(t)dt = Gg fy(t)dt, 
the disturbance estimate is proportional to the integral of the state estimation error. 



2.6.2.4 Example: Use of a Disturbance Functional Observer for 
the Control and Simultaneous Dry Friction Cancellation 

The present application, which may concern any mechanical system with an actua- 
tor submitted to dry friction, has been studied on an inverted pendulum by 
[BaB088] and will be handled numerically in the solved exercises at the end of 
this chapter. It constitutes also a transposition to the discrete-time systems of the 
material which has been presented previously in the continuous case. 

Its principle consists in representing the dry friction force f c as a constant dis- 
turbance, applied to the unique control input u of the plant. Its amplitude is un- 
known and its sign changes at each change of direction of the cart carrying the in- 
verted pendulum. The state equation of the continuous time plant is thus: 

x = Ax + b(M + / c ). 

The friction is modeled by the equations (2.73), with A; = and Ce = I , and 
thus also f c = E, and E = b . 
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An augmented model of the plant and the disturbance is built as in 
Sect. 2.6.2.2, according to (2.74), and represented by the matrices A , b and C 
defined below: 
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(2.78) 



An observer is designed then for this augmented system. The disturbance step 
jumps will be interpreted by the observer as sudden changes of the initial condi- 
tions of £ in the integration of the state equation £, = , and the observer will 
undergo each time simply a new transient response. 

Let us switch now to the discrete model of the plant, sampled at a period T s . 
Initial plant: 



x t+i=^ x t+Y"t 



y t =cx t 



with 
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= e ATs ; y= f 'e Ar 'b^. 
Jo 



Augmented model: 
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The control law will have the form: 



**+JOr.*-/ c . 



14 J 



-JO,.*. 



(2.79) 



with f c k =E, k . Suppose the plant is of 4 th order and only the state variables x { 

and x 2 are measured. Denoting e k the part of u k which is not accessible to the 
measurement, namely the following linear combination of the state variables 
x 3 and x 4 and of the disturbance % k : 



e k =-(0 £, £ 4 1 



14 J 



a 



14 J 



(2.80) 
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of which the observer will yield an estimate £ k , the control law (2.79) becomes: 

u k = -?i x i,k-?2 x 2,k + £k+ M y r ,k ■ 

The functional observer will have the state representation 

fo k+1 =Fz k +Tyu k + gy k 
1 e k =h T z k + ky k 

where F, T and g are constant, real matrices and vector, satisfying: 

TO-FT = gc T , 

and where h T and k satisfy: h T T + k c T = a T . 



2.7 Solved Exercises 

Some exercises of this chapter are the sequel of exercises which have been solved 
at the end of Chap. 1. More details can be found there. 



Exercise 2.1 Control of an Inverted Pendulum (contd) 

Consider once more the inverted pendulum LIP 100 (Amira). We wish this time 
to compensate the absence of some state variables among the measurements by 
implementing an observer. This exercise will be solved in continuous time, with- 
out integral action. The reader is invited to repeat this solving by using the discrete 
time model. 

a) Repeat the design of a control law, with the same imposed poles as in Ex- 
ercise 1.1, but by choosing this time another design method than pole 
placement ("acker.m" or "place. m"), and compare the obtained control 
law with the previous one. 

b) Determine, in the case where only the cart position is measured, a full-state 
(Luenberger) observer having a unique eigenvalue of order n = 4 at 
s = -6. 

c) Determine a reduced-order observer in the case where the two measure- 
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merits mentioned in Exercise 1.1 are available, by imposing again to this 
observer a unique eigenvalue s = — 6 . 

d) Test the control system determined above, with the two observer types suc- 
cessively, by simulation with the Simulink® software, and by selecting dif- 
ferent initial states for the plant and the observer. 

e) What happens once the observer initial transient response has decayed? 
Examine this by applying a square signal as a reference in the simulation 
diagram. 

Solution: 

(a) Run the MMCE.m program, with the plant inverted pendulum LIP100 
(Amira), with two measurements, then Choice of model: continuous, Syn- 
thesis of state-feedback (default), Integral action: = without (default), 
Component of y to be set to its reference value: 1. The design methods sim- 
ple modal control and decoupling method do not apply here, the first because it 
allows shifting only one single eigenvalue, the second because it applies only to 
plants having identical number of inputs and outputs. The complete modal syn- 
thesis (Roppenecker) could be used, but at the condition to separate slightly the 
4 prescribed eigenvalues since this method does not accept a number of identical 
eigenvalues larger than the number of plant control inputs. 

Therefore, select here 5) general formula (Becker-Ostertag), with the con- 
trollability canonical form for the closed-loop system and A Ll =-5s _1 , 
i = 1, • • • , 4 . The diagonal canonical form would not be adapted here, since it 

makes the general formula identical to the Roppenecker' s formula and since we 
want to introduce a multiple eigenvalue of multiplicity order larger than the num- 
ber of plant inputs (see Remarks of Sect. 1.7.1). Exactly the same control law as 
the one of Exercise 1.1 is obtained here. 

(b) Returning to the Main menu, select the full-state (Luenberger) observer 
design. The pole placement is not applicable here, since we want to place 4 ob- 
server poles, whereas the plant has only 2 outputs. We can on the contrary select 
again the general formula, with controllability canonical form, which does not 

have this restriction, and the following parameter vectors: [l l] , [0 l] , [l 
and [l — 2] . The observer is calculated in the generalized observer form, with 



T 



G = 



9.61 8.951 

12.65 12.34 

17.58 18.06 

218.30 221.14 



F=A-GC = 



-9.61 -8.95 -1.950 

-12.65 -12.34 1 

-17.58 -18.19 -1.915 0.0008 

-218.30 -199.66 26.339 -0.136 
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(c) From the Main menu, select this time the reduced-order observer design 
and the Shape of C: (l q | 0). It is possible here to synthesize the observer gain 
matrix G r by pole placement since there are only 2 poles to prescribe, the re- 
duced-order observer being of order n — q = 2. We get: 



-2.095 0.001 ) 
-13.505 5.864 



F = A 22 -G r A 12 = 



-6 ) 
-6 



with J = TB = B 2 - G r Bj according to (2.42) and H and K given by (2.49). 

(d) The simulation diagrams proposed in Appendix D allow connecting the ob- 
server in the loop or leaving it unused, so that we can study its dynamical behavior 
independently from that of the control loop, by closing the loop on the state. The 

free responses from the initial states x = (0 —2 0.5 0) and z = , which 

are given in Fig. 2.1 1, have been recorded with the two types of observer acting in 
free wheel, thus out of the loop. 
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Fig. 2.11 Free responses of real and estimated states, in case of: 
(a) a full-state observer (order 4); (b) a reduced-order observer (order 2). 
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The comparison of the state variable x 3 plots in the two sides of the figure 
shows that the reconstruction of the reduced-order observer converges faster to the 
real state than that of the «th-order observer. This result was expected, since the 
first observer has two poles less than the second at the same values s = — 6 . This 
constitutes an advantage of the reduced-order observer, its drawback being that it 
does not offer anymore the possibility of comparing estimated and measured val- 
ues, which is at the base of the instrument failure detection methods by analytical 
redundancy. 

The plots concerning the state variable x 2 in the case of the reduced-order ob- 
server are undistinguishable right from the initial time. This results from the fact 
that this state, just as x i , is no longer reconstructed by this observer but simply 
transmitted from the measurement y 2 = x 2 . 



(e) Let us design now deliberately an identity observer much slower than the 
closed-loop system, so as to illustrate better its influence. With a choice of all its 4 
poles located at s = —2, the design by the general formula, with the controlla- 



bility canonical form, and arbitrary parameter vectors [0 l] 
[l — 2] , yields the matrices: 



[1 0] T , [0 0] and 



0.380 0.409 

-3.530 5.569 

0.050 -0.140 

-13.284 29.568 



F = A GC„ 



-0.380 
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-1.950 
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0.011 


-1.915 


0.001 


13.284 


-8.095 


26.339 


-0.136 



with J = B , whereas H and K are given by (2.50). Fig. 2.12 shows the response 
of the system, closed through this observer, to a square wave reference of unit am- 
plitude, from an initial plant state x = (0 —2 0.5 0) different from that of 
the observer z = . 
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Fig. 2.12 Closed-loop square wave response for unequal plant and observer initial states. 
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This figure illustrates well the fact that, once the estimated state has converged 
to the real state, the observer dynamics does not appear anymore in the response to 
reference steps: the observer poles have no more effect. However, a new distur- 
bance of the plant initial state would trigger a new transient response of the ob- 
server, with its own dynamics (see comments of Sect. 2.5.5). 



Exercise 2.2 Dry Friction Compensation by a 
Disturbance Observer. 

Consider again the inverted pendulum LIP 100 (Amira). The goal is here to 
compensate the dry friction of the cart on its rail by means of a disturbance ob- 
server. This study will be performed in discrete time. 

a) Assume here that the plant has three measurements: x c , and x c . Calcu- 
late first a control law for the discrete-time model of this plant, by placing 
all closed-loop continuous eigenvalues at X Ll . = — 5 s _1 , / = 1, • • • , 4 . 

b) Calculate a reduced-order observer, which allows on one hand to recon- 
struct the missing state variable, x 4 = 9 , on the other hand to compensate 

the dry friction, which will be modeled as a constant disturbance, of un- 
known amplitude and changing sign at each direction change of the cart. 

c) Simulate the closed loop by including this disturbance in the diagram, and 
evaluate the efficiency of the solution developed in question (b). 

Solution: 

(a) The plant is equipped here with the three-measurement matrix 

(l o o o s 

10 0, which should be substituted in the program MMCE.m to the 
1 0J 

default measurement matrix by the statement change to three measurements. 

The control law calculated by the pole placement method is obviously the 
same as the one calculated in question (d) of Exercise 1.1, since it does not depend 
on the number of state variables accessible to measurement: 

L = £ T = (-2.0298 2.0162 3.5714 0.4434), M = -2.0298. 

(b) The disturbance observer will be calculated as in Sects. 2.6.2.3 and 2.6.2.4, 
where a functional observer will be replaced by a reduced-order observer. 
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The MMCE.m program allows performing these calculations by choosing in the 
Main menu, the synthesis of a disturbance observer, option available only 
with the inverted pendulum and in discrete time. The program does the following: 
first, the continuous-time disturbance-observer model is incorporated into the con- 
tinuous-time plant model, as in Sect. 2.6.2.4; then the augmented system is sam- 
pled at the period T s = 0.03 s . 

The continuous-time model augmented by the constant disturbance has, accord- 
ing to (2.78), the state representation: 
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This augmented model is then sampled with zero-order hold by the statement 
c2dm(A_tild,B_tild,C_tild,D_tild,Ts,'zoh'). The result is the discrete time state 
representation below, which is then partitioned as shown, since the state variables 
accessible to measurement are the first three ones: 



O 



1 0.0001 -0.0569 
1.0097 0.0116 
-0.0038 0.9442 



0.6434 
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At the difference to Sect. 2.6.2.4, instead of estimating only a linear combina- 
tion of part of the state variables and of the disturbance, as in (2.80), the reduced- 
order observer will have the duty to estimate the missing part of the augmented 

state vector ix k ^ K \ of dimension 5. The control law, applied to that partially 



reconstructed state vector, will then be 



(4j 



-\l x t 2 «3 £4 1 



16 



The design of this reduced-order observer, of order n — q = 5 — 3 = 2, by pole 
placement, with two poles at —10 s _1 , yields the following matrices for its gener- 
alized state model: 
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(c) The simulation is made by means of the third simulation diagram of Appen- 
dix D. In this diagram, the observer is broken down into the various matrices 
which compose its representation in the generalized observer form. A Coulomb & 
Viscous Friction block serves to model the dry friction, the viscous friction coef- 
ficient being set to zero there. On the real setup where the experiments have been 
performed, the dry friction represents a threshold of about 2.5 V at the point where 
the control voltage is applied to the plant. 

The switch S 2 of the diagram is used to enable or disable this disturbance term, 
while the switches S 3 and S 4 allow choosing between compensation by distur- 
bance observer, compensation by fixed threshold with sign switched at each cart 
travel direction change, and no compensation at all. 

The two oscillograms in Fig. 2.13, recorded with a square wave reference sig- 
nal of 2 volts amplitude and the same scales, point out the limit cycle resulting 
from the nonlinearity constituted by the dry friction. The substantial gain brought 
by the compensation by observer can be seen. It is clear that the fixed threshold 
compensation is perfect only if the intensity of the dry friction is known exactly, 
and if it does not evolve with time, situations rarely encountered in practice. 





Fig. 2.13 Closed-loop square wave response of the cart position, 
(a) without dry friction disturbance compensation, (b) with disturbance observer. 
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In this chapter and the following, a completely different approach will be taken 
than in the previous chapters: instead of considering the eigenvalues of the closed- 
loop control system or those of an observer, we will invoke quadratic optimization 
methods, deterministic here, stochastic in Chapter 4. For this reason, the dynamic 
behavior of the control system is no longer mastered directly. Fortunately how- 
ever, the control designed in this chapter will have robustness properties, going 
thus beyond the simple stability requirement, as will be seen at the end of the 
chapter. 

At the difference with other books using the dynamic programming approach 
and the Bellmann's Principle of Optimality, as in [AlSa04], the approach taken in 
the present book to establish the optimal control law will be based on the calculus 
of variations and the Lagrangian multipliers, which are the mathematical bases for 
many optimization problems in Physics and will allow us to make a parallel with 
the equations of motion of Analytical Mechanics. The correspondence with the 
dynamic programming will be discussed at the end of this chapter. The calculus of 
variations is also the method used in [Gee07], where closed-loop state-feedback 
optimal control laws are derived directly from the Bellman's principle in one step. 



3.1 Introduction 

In control theory, there are two different kinds of optimization: 

• the parametric optimization, which consists in looking for the optimal parame- 
ters of a controller C{s) of imposed structure, e.g. the proportional and integral 
coefficients K and T t of a PI controller, by trying to minimize a function J of 

these parameters, as e.g. J = \ e (f)dt ; one tries then to find K and T i 

dJ „ , dJ 

such that = and = ; 

dK„ dT, 
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• the functional optimization, where it is assumed on the contrary that the con- 
troller structure is completely free and thus that the control law u(7) or u^ 
which will be applied to the plant input can be chosen arbitrarily. To a given 
function u(7) or u^ is associated a given value of some criterion J, which is 
thus afunctional. By language abuse, J is often called simply a cost function. 

It is this second kind of optimization which will be the topic of this chapter. The 
objective is to find the, function u(t) , respectively u(7) in the M1MO case, which 

minimizes the criterion J, which is always scalar and which depends on this func- 
tion, hence the appellation functional, or structural, optimization. 

Mathematical fundamentals. The variational calculus has been introduced by 
the Bernoulli brothers, who studied with this mathematical tool the well known 
brachistochrone problem, in 1697. 

In the 1 8th century, Euler and Lagrange established the basics of variational 
calculus. The Euler-Lagrange equation was used later mainly in Physics, in the 
fields of Optics and Mechanics: Fermat's Principle for the propagation of light in 
media with different refringencies, Least Action Principle of Maupertuis (18th 
century) and of Hamilton for the determination of motions in Analytical Mechan- 
ics, in the 19th century. The work of Hamilton in Analytical Mechanics has been 
continued later by Jacobi. 

The technical development of the 20* century imposed the taking into account 
of constraints. Let us mention the works of Bellman (1950), of Feldbaum (1953) 
and of Pontryagin (1956), if only the most important milestones should be high- 
lighted. 



3.2 Free Final State Optimization 
3.2.1 Problem Description 

This is the basic problem of Optimal Control. The hypotheses are the following. 
Consider 

• a non necessarily linear, nor time -invariant system: 

i(0 = f[x(f),u(0,f], (3-1) 

x (°) = x o S iven > 
u(0 € "2/ ; 
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• and a cost functional: 

J = h[x(t f )} + J"' f [x(t)Mt),t}dt , (3.2) 

where t, is the final time, which is fixed here, and J, h and f are scalars. 

The objective is to find the control law u(t) , in the universe ^U of all possible 

control laws, which minimizes J. The choice of this cost functional results as a 
matter of fact from the combination of several criteria [F6R088]. 

The integral term expresses the desire to minimize, all along the trajectory, the 
energy consumed by the plant, at the input of which the control law u(t) will be 
applied, the unnecessary deviations from its state space trajectory between the ini- 
tial state x and the expected final state x(7,) , deviations which affect of course 

the variations of the output y(t) = C x(t) on this path , and the time t,- itself. 

Very often, as will be seen in more details in Sect. 3.5, the used criteria are of 
quadratic type, so as to take into account variations of u(t) or \(t) of opposite 
signs equally. Therefore, energy and optimal trajectory criteria will often enter 
into the previous f function as two quadratic forms, yielding thus a cost func- 
tional having the following form: 



/ = - f (x T Qx + u T Ru)<A. 

2 Jo 



The factor 1/2 is not important and facilitates only the subsequent calculations. 

A minimization of the trajectory duration, thus of tr , can also be written in the 
form of an integral criterion: 

J = I ' hdt. 



J" 

J 



The two previous criteria, quadratic criterion and time optimality, are grouped 
in the more general form, called Lagrangian cost functional: 



J= f ' f [x(t),u(t),t}dt, 
*J 



which represents the second term of the cost functional (3.2), and which will be 
encountered again in the case of optimization problems with entirely fixed final 
state, in Sect. 3.3. 

It remains to justify the first term of expression (3.2). At the difference to the 
previous one, this criterion deals only with the final state. It intervenes in prob- 
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lems where the final state is crucial, such as the hit of a projectile on a moving tar- 
get or the accuracy of moon landing, as far as the height coordinate of the Lunar 
Module above the landing point is concerned. In its most general form, namely 

J = h\\(tj-),tj- 

this criterion is called Mayer criterion, from the name of a mathematician of the 
19 th century end. 

The cost functional resulting from the grouping of the two previous criteria is 
called cost functional of Bolza, a mathematician of the beginning of the 20* cen- 
tury, and constitutes very closely the cost functional (3.2), which will be used in 
this whole chapter. 



3.2.2 Modified Cost Functional. Hamiltonian 

The problem encountered here is typical of variational calculus. In order to char- 
acterize an optimal control, one tries to evaluate the effect of a small variation of 
u(t) on the cost functional and to impose that it does not produce any diminution 

of the last. The approach will thus have three steps: 

1 . start from a given control law u(t) , assumed optimal; 

2. let u(7) vary slightly and calculate the corresponding variation of the cost func- 
tional J; 

3. this variation should be positive (non decreasing) if the starting control law 
u(t) was optimal, thus if J was minimal. 



3.2.2.1 Method of Lagrange Multipliers 

Due to the existence of an auxiliary condition, namely the differential equation 
(3.1), which can be written as 

i(t)-f[x(t),u(t),t]=o, 

and which causes any variation of u(t) to result in a variation of x(t) , it is not 

possible to solve this problem in a direct way. 

The problem is then solved indirectly by the method of Lagrange multipliers, 
which allows transforming an extremum problem with auxiliary condition to an 
extremum problem without auxiliary condition. 
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The multiplier considered here is a vector of same dimension as x and depend- 
ing on the time: y(t) = (^ (t) •■• ip„ (0) • 

The method consists in adding to the integrand a permanently vanishing term, 
which contains the Lagrange multipliers. The following modified cost functional, 

J = h[x(t f )) + f' f f [x(t)Mtlt]dt+f tf y T (t){i(t)-f[x(t)Mtlt]}dt,Q.3) 

is thus built, which is identical to the previous one. Note that the auxiliary condi- 
tion is now included in J. It is thus no more necessary to account for it separately. 



3.2.2.2 Hamiltonian 

In order to shorten the equations, one introduces the Hamiltonian: 

H(y,x,u,t) = -f (x,u,t) + y T -f(x,u,t) . (3.4) 

Remark 3.1. H is a scalar function of x\(t) , of x(t) , of a presently unknown vec- 
tor \|/(7) , which will be defined later, and also, in the case for instance of time- 
varying systems, an explicit function of time. 
By substituting (3.4) into (3.3), we get: 

J = h[x(t f )] + J''{\ V T (t)Mt)-H[\ V (t)Mt)Mt),t}}dt. (3.5) 

Remark 3.2. For time-invariant systems, the explicit time dependence disappears 
from the state equation: x(t) = f [x(?),u(?)] . If, in addition, the integrand f of the 

minimization criterion does not depend explicitly on time, the explicit time de- 
pendence disappears also from the two following equations: 

/o = /o[ x (0,u(0], 

H = H{y(t),x(t)Mt)]- 

This brings about a simplification of the writings, which will be used in the rest 
of this chapter. For the same purpose, the time dependencies of the functions 
which are arguments of the above expressions will also be omitted, as long as no 
ambiguity results: x = f(x,u), / =/ (x,u), H = H(\\i,x,u) . 
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3.2.3 Variational Calculus 



As a starting point, suppose that a given nominal control law u(7) , which satisfies 
the constraints u(7) € HI , has been found. This control u(7) produces some cor- 
responding state trajectory, x(t) . 

To this control u(7) a small variation, or perturbation, 8u(7) , is then applied, 
which transforms from that time on the initial control law to u(t) + 8u(/) . This in- 
finitesimally small variation is such that: 



/ \bu t (t)\dt<i 



(3.6) 



whatever e , arbitrarily small, and this for all components of u, thus for every i. 
The perturbation on u(7) causes obviously a small variation (perturbation) of 

the system state trajectory, which becomes from that time on x(t) + dx(t), as il- 
lustrated in Fig. 3.1 in the case of a one -dimensional problem. 



u(0 + 8u(f) 




Fig. 3.1 Perturbed control law and resulting trajectory. 

The trajectory perturbation, 8x(£) , is itself an infinitesimally small quantity of 
first order in e for every t, as can be seen by applying (3.6) since the state de- 
pends essentially on the integral of the control law. 

The corresponding cost functional variation amounts to 

bJ = h[x(t f ) + dx(t f )^-h[x(t f )\ 

+ f / {v|/ T 8x-//[v|/,x + 8x,u + 8u] + //[v|/,x,u]}^. (3.7) 
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The first term of the integral is evaluated by integration by parts: 

I \\i 1 (t)8x(t)dt = \\i 1 (t)8x(t)\ - I \\i J (t)Sx(t)dt 
Jo lo Jo 

= v|/ T (t f ) hx{t f ) - y T (0) 5x(0) - f ' y T 8x dt . (3.8) 

To calculate the second term of the integral, we will use its Taylor series ex- 
pansion and its first order approximation, i.e. at the order of the infinitesimally 
small e introduced above. Let us first visualize separately the two variations, by 
subtracting from the integrand, and adding to it, the term H(\\i, x, u + 8u) : 

/•t f 
[H (v|/, x + 8x, u + 5u) — 7f(v|/,x,u)] dt = 
o 

r'f 

I [//(v|/,x + 8x,u + 8u)-7f(v|/,x,u + 8u) + if(v|/,x,u + 8u)-//(v|/,x,u)]A.(3.9) 

i/ o 

Apply the first order Taylor series expansion approximation to the first two terms 
of the integrand. The following approximation holds for the previous integral: 



r'f Id 

I — — H(\\i,\, u + 8u) 8x + //(\|/,x, u + 8u) — H(\\i,\,u 

Jo [dx l 



dt . 



Subtract now from the integrand, and add immediately again to it, the term 
dH(\\i,x, u)/<9x T 8x . The above integral reads thus: 



r'f\\d d 1 

I — - Y H(\]i,x,u + du)-— T H(^,x,u) 6x 
<J Wax n\ 



or: 



— H(\\i,x, u) 8x + H(\\i,x, u + 8u) — H(\\i,x,u)\ dt , 
dx 



C'f\d d 

I — -H(\y,x, u + 8u) =rH(\y,x,u)\dt\5x 

Jo [dx dx J J 

r'/ld 

+ / — -i/(i|/,x, u)8x + //(\|/,x, u + 8u)-//(\|/,x,u) \dt . (3.10) 
•^o [dx 1 



The first of these two integrals is an infinitesimally small quantity of first order 
in e , for the reasons invoked above, concerning the effect of the control perturba- 
tion on the state trajectory, by taking into account in addition to the hypothesis 
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(3.6) the expression (3.4) of the Hamiltonian. Since this integral is multiplied to 
the right by 8x , itself infinitesimally small of first order in e , as was proved 

above, the product of these two factors is thus infinitesimally small of the order of 
e 2 , which makes it negligible compared to the second integral of (3.10). 
We obtain thus for the expression (3.9) the following approximation: 

r't 

\ [H(\p, x + Sx, u + 8u) — H(\y,x,u)]dt 

r'rld 1 

— / — — H(\\i,x, u) dx + H(\\i,x, u + 8u) H(\\i,x,u) \dt . (3.11) 
Jo \8x 



Substituting (3.8) and (3.11) into (3.7), the following expression is finally ob- 
tained for the cost functional variation: 



SJ: 



dx(t f ) + v|/ T (t f ) 5x(t f ) - v|/ T (0) Sx(0) - f '' y T 5x dt 

if 

r'i\d 
- I — ~ H(\\i, x, u) 8x + H(\\i, x, u + 8u) — H(\\i, x, u^ 

Jo \d\ 



After some rearrangement of the terms and the use of (B.7) of Appendix B, this 
expression is also written: 



5/: 



d h 



dx 



+ v(t f ) 



8xfi?f 



8x( f/ ) - i|/ T (0)8x(0) - f '' L + |-//(v|/,x,u) 

Jo [ ax 

- f '[//(v|/,x,u + 8u)-i/(v|/,x,u)]c?f.(3.12) 

o 



Note first that 8x(0) = , since a control variation does not modify the initial 

state. The second term of the right side of (3.12) disappears thus. 

The goal of the present calculation being to find an extremum of our cost func- 
tional, we want to realize hJ = . 



3.2.4 Adjoint Equation 

Since the vector \|/(?) is still arbitrary at this state of the calculation, let us choose 

it so as to cancel the first two terms on the right side of (3.12). It is enough for this 
to choose v[(t) as the solution of ^the adjoint differential equation 
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y(f) = ~H(y,T,u) , (3.13) 

ox 

which satisfies the final condition (also called transversality condition): 

V(t f ) = ~ . (3.14) 

dx 

Remark 3.3. With (3.4) and (B.16) the differential equation (3.13) becomes 

Bf r)f T 

-V(0 = -^- + ^-V(0 • (3-15) 

ax ax 

This relation constitutes a system of n linear differential equations which, with 
the final conditions (3.14), allows determining the n components of \y(t) . Note 
that, due to the presence of a minus sign at the left side, their numerical solving 
will occur backwards, i.e. at decreasing time, starting from the final conditions. 

Remark 3.4. The vector \y(t) is called costate vector. 



3.2.5 Hamilton 's Equations 

The Hamiltonian definition (3.4) and the formula of differentiation (B.8) show 
readily that 

^ = A (v T. f) = f 

d\\i d\\i 



It follows, according to (3.1), that 



dx . dH 

— = x = . (3.16) 

dt d\\i 



This equation, together with (3.13) repeated here: 



dy ■ dH 
dt ox 



represent the Hamilton's (conjugated) equations, well known in Analytical Me- 
chanics. 
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3.2.6 Similarity between Analytical Mechanics and 
Optimal Control 



Obviously a full similarity exists between the extremum problem tackled here and 
the Hamilton 's Principle of the Analytical Mechanics, which states that, among all 
motions transferring a given system from a point M of its configuration space at 

initial time t to a point M l at time t t , the real motion is the one for which the 

action integral 1=1 Ldt is extremum, i.e. 8/ = 0, where L is the system's 
J t 

Lagrangian. Table 3.1 illustrates this formal similarity. 

Table 3.1 Formal similarity between Analytical Mechanics and Optimal Control. 



Analytical Mechanics 


Optimal Control 


Generalized coordinates of n mass points: 1 


System state variables and state vector: 


q = {<?,}, 7 = 1,...,/ 


x = {x I }, i = \,...,n 


Lagrangian: 


Function involved in the cost criterion J: 


L 


/o( x > u >0 


Integral (criterion) to minimize: 


Cost functional (for fixed final state): 


/= V Ldt 


J= f ' f [x(t)Mt),t]dt 

*J 


System's Hamiltonian: 


System's Hamiltonian: 


H = -L + '£p J q J 


H = -/ (x,u,0 + V r (0"f(x,u,0 

n 


7 = 1 


= -/ (x,u,0 + X^*; 


Canonical conjugated momentum of q . : 


Costate vector: 


p = {p J },j = i t ...,f 


\|/ = {?/>.}, i = \,...,n 


Conjugated Hamilton's equations: 


Conjugated Hamilton's equations: 




dH 

dp j 




' . dH 

x = 

<9\|/ 




dH 
dqj 




dH 
w = 

a x 





'.The number /of generalized coordinates is the number of degrees of freedom of the set of n 
mass points, i.e. the number of independent coordinates remaining once the constraints (links be- 
tween points) have been taken into account 
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3.2. 7 Pontryagin 's Maximum Principle (1956) 

Assuming that the hypotheses introduced by (3.13) and (3.14) are satisfied, the 
cost functional variation bJ , given by (3.12), becomes: 

8J~- f y [i/(\|/,x,u + 8u)-#(\|/,x,u)]A. (3.18) 

The condition of optimality is then the following: for the optimal control law u, 
the Hamiltonian H is maximum, in other words 

#[\|/,x,u + 8u]<#[\|/,x,u], (3.19) 

Wefo.fJ and V(u + 5u)eW. 



Proof by Contradiction. Suppose that, for some value of?, there exists a perturbed 
control law, (u + 8u) e Hi , such that 

H [v(0> x (0, u(0 + 8u(0] > H [i|/(0, x(0, u(0] • 

One could then modify the function u, as indicated in Fig. 3.1, in such a way that 
the integrand of (3.18) be positive in a small interval containing this value of t. 
This would result in a 8 J < , which would contradict the fact that the function u 

produces the minimum value of the cost functional J. 

In conclusion, for every t, the particular value u(t) which corresponds to the 

optimal control law has the property to maximize the Hamiltonian. This result 
constitutes the Pontryagin 's Maximum Principle for this problem. 



3.2.8 Recapitulation: Theorem (Maximum's Principle) 

Assume that x\(t) E H/ and x(t) represent respectively the optimal control law 
and the corresponding state trajectory, solutions of the optimal control problem 
defined at the beginning of this section. There exists then an adjoint trajectory 
\|/(?) such that the following holds, simultaneously: 

\(t) = f [x(?),u(0] (plant state equation) 
x(0) = x (plant initial state) 
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df df T 

— \|/(0 = H • \y(t) (adjoint state equation) 

dx dx 



dh\x(t f )\ 
\y(t f ) = — ■ — - (transversality condition) 

dx 

yte[o,t f \ WV(u + 5u)e^: 

H [ y(f ), x(0, u(0 + 8u(0] < H [y(f ), x(t), u(tj\ . 

This inequality is the maximum 's condition, where H is the Hamiltonian: 

//(i|/,x,u) = -f (x,u) + \|/ T f (x,u) . 

Remark 3.5. In some books, such as [Gee07], this result is given as the Pon- 
tryagin's minimum principle. This is simply due to the fact that these authors in- 
troduce a Hamiltonian with opposite sign as compared with our definition, their 
function f and their costate vector \|/ having opposite signs to ours. The x value 

which maximizes a function y(x) is obviously the same which minimizes — y(x) . 

3.3 Problems with Final State Constraints 

In Sect. 3.2 we have assumed that the final state was entirely arbitrary: the tackled 
situation was called therefore free final state problem. 

In many cases, however, the final state is confined to some domain of the state 
space. In this case, the previous problem hypotheses will be completed by a set of 
constraints on the final state: 

x i(tf) = x i, i = l,---,r, (3.20) 

where the plant state variables have been renumbered if needed so as to place at 
the beginning the r ones which are submitted to constraints. 

The search for conditions which must be satisfied by an optimal solution to this 
problem will occur by following the same approach as in Sect. 3.2. The modified 
cost functional J is created again, and the variation induced on J by a change of 
the control law from u(^) to a perturbed law u(t) + 8u(?) which satisfies also all 

the constraints, is evaluated. 

Let us repeat here this variation, which has been established in (3.12): 



3.3 Problems with Final State Constraints 
iT 
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dh 



dx 



\\i(t f ) 8x(^)-\|/ (O)Sx(O)- I \|/H H(\\i,\,u)\ Sxdt 

Jo [ dx \ 

r'r 
— J [i/(\|/,x, u + 8u) — H(\p,x,u)]dt . 
J o 



Here again, 8x(0) = , and again we will choose i|/(7) so as to cancel as many 

terms as possible on the right side of this expression. Therefore we will impose 
again, as in (3.13), 



v(0 = 



d_ 

c)x 



H(y,x,u), 



but the specification of the boundary conditions will be somewhat different from 
(3.14). 

Indeed, since x t {t f ) is fixed for i = \,...,r , it ensues that 8x ; (^) = for 

these same values of index ; for all the trajectories involved. It is thus not neces- 
sary to impose the condition (3.14) to all the components of Vf{t.) to cancel the 

first term of (3.12). It will be enough to impose 



MY- 



dh 



dx 



dh_ 

3X; 



i = r + 1, . . . , n , 



(3.21) 



to ensure that the dot product of the first term of 8J vanishes, i.e. to obtain 



dh 



dx 



Mt f ) 



8x(?,) = 0. 



To summarize, the rule is the following: 

• if x { (tf) is submitted to constraint, V;(*/) is free; 

• if x { (tf) is free, ip t (tj-) is submitted to constraint. 

Remark 3.6. If x(t,) is entirely fixed, Vf{t,) is entirely free. The transversality 

condition does therefore not apply anymore, and the term weighting the final state 

in the cost functional, namely h \x(tf)\ , which is then constant, has no reason to 

stay there and can be suppressed from the expression (3.2) of J, which reduces 
thus to: 
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J = J^' f [x(t)Mt),t]dt. (3.22) 

3.4 Problems with Free Final Time 

In some problems, the final time te is not fixed and can be chosen itself so as to 

yield the smallest possible cost functional. As an example, let us mention the tra- 
jectory of a moon rocket which minimizes the fuel consumption: there is no rea- 
son in this case to specify the landing instant. 

If the final time te is left free, the optimization problem consists now in find- 
ing simultaneously te > and u(7), < t < t, , such that the cost functional J 

given by (3.2) is minimal. 

It is clear that, if the best te value were known, one could impose it and would 

be left with the previous problem. All the conditions listed in the maximum's 
principle theorem must thus hold here in the same way. Only an additional condi- 
tion is required here, from which the unknown value of t ,- will be derived. 

Consider as previously a modification of the control law u(t) to a new law, 
u(t) + 8u(?) , which will produce a new trajectory x(t) + bx(t) , but this time with 
a new final time t f + dt t . 

The new aspect of this problem is that the system final state variation is no 
longer hx(te) , since the final time itself varies. The new final state is in reality 

\(t f ) + 5\(t f ■+ dtj ■) , and, if we denote it with x(tj) + d\(t f ) , the following 

holds at first order approximation: 

dx(t f )~5x(t f ) + x(t f )dt f =Sx(t f ) + f\x(t f ),u(t f )]dt f . (3.23) 

A calculation similar to the one performed in Sect. 3.2 would lead to the fol- 
lowing expression for the cost functional variation: 

dh 



8J~ — dx(t f ) + \\i l (t f )5x(t f )-\\i l (0)dx(0) + f \x(t f ),n(t f )\dt f 

\|/H //(v|/,x,u) Sxdt— j [//(\|/,x, u + 8u) — H(\y,x,u)]dt , 

dx \ Jo 

which replaces (3.12). 
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The additional term f () \x(t j-),\x{t f )\dt f , which is underlined in the new ex- 
pression, stems from the fact that, in the first order Taylor series expansion, the 
variation dtf is also taken into account this time. Now, the explicit dependence of 

Jon t f is obtained from its partial derivative dJ/dtf , which according to (3.3) 
amounts to 



_d_ 

dt f 



J o ' {/o (x,u) + v|/ T [i - f (x,a)]}dt = f \x(t f ),a(t f )} , 



where the formula for differentiation of a definite integral, whose limits are func- 
tions of the differential variable, (Leibnitz integral rule) has been used: 



d_ 
dx 



<p(x,y)dy\= I 

a(x) ! it a(x) 



dip , db r , . . i da r , . 
-^—dy + — <p\x,b(x)\- — <p\x,a(x) 

(x) ox dx dx 



By choosing again \|/(?) so as to satisfy the adjoint equation (3.13), by noting 
again also that 8x(0) = , and by making the substitution 

8x(t f ) = dx(t f )-f \x{t f \u{t f )\dt f 

derived from (3.23), we obtain: 



8/: 



dh 



dx 



V(tf) 



d x{t f ) - [ y T (t f ) f [x(t f ), u(t f )]-f [x(t f ), u(t f )]} dt f 
— I [H(\\i,x, u + 8u) — H(\\i,x,u)] dt . 

*J 



The first term is cancelled by imposing to ip t (t) the same conditions as in the 
problem with fixed final time, equations (3.14) or (3.21) in the case of partially 
constrained final state. 

The cancellation of the integral term leads to the usual condition of the maxi- 
mum ofH. 

The only term remaining to cancel is thus the second one. Now: 

-f [x(t f ),u(t f )} + ^ J (t f )-f[x(t f ),u(t f )} = H[^(t f ),x(tf)Mt f )}- 



The optimal t f value will thus correspond to 
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H[^(t f ),x(tf)Mtf)\ = . (3.24) 

This equation represents the supplementary condition required to determine the 
supplementary unknown t f . 

3.5 Linear Quadratic Control (LQC) 

This control law, also called Linear Quadratic Controller, or LQ Controller, is well 
known under its abbreviation LQC. The apparent contradiction between these two 
adjectives will be clarified hereafter. 



3.5.1 Problem Formulation 

The optimal control theory presented previously is now applied to linear, not nec- 
essarily time -invariant, systems: 

i(0 = A(0x(0 + B(0u(0 

y(0 = C(0x(0 "'"" 

with a cost functional, or criterion, of quadratic nature, as seen in Sect. 3.2.1: 

J = -x T (t f )S(t)x(t f ) + - f' \x T (t)Q(t)x(t) + u T (t)R(t)u(t)\dt , (3.26) 
2 2 v fo 

The time dependencies of the elements involved in these two formulae will be 
omitted temporarily to simplify the writings, and will be restored after the inter- 
mediate mathematical developments. 

A few comments can be made about the criterion J. Its second term will serve to 
minimize the energy and the trajectory, as said in Sect. 3.2.1. 

R is a symmetric, positive definite matrix, since it represents a positive definite 
quadratic form, which means (see Sect. B.6, Appendix B) that: 

u T Ru > 0, Vu^O; u T Ru = for u = only . 

This appears clearly in the relatively frequent situation of a diagonal matrix R, by 
considering that all the terms in u] of the quadratic form representing the influ- 
ence of the energy consumption are of course weighted positively. 
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The Q matrix, on the other hand, is a symmetric, positive semidefinite matrix. It 
represents thus a positive semidefinite quadratic form: 

x T Q x > 0, V x ; =>■ x T Q x can be zero for x * . 

If indeed we consider the system output, y = Cx , and a weighting of the trajec- 
tory followed by this quantity by means of a positive definite quadratic form of 
matrix G, 

y T G y = x T C T GC x = x T Q x , 



-iT/ 



where Q = C GC , this quadratic form can vanish without x being zero, accord- 
ing to the shape of the C matrix. As an example, assume that the two first columns 
of C are identical; the nullity of y and thus of x T Qx will be ensured for arbitrary 
x 2 = — Xj and x 3 = • • • = x n = . 

The role of the first term of (3.26) is explained as follows. The usually wished 
final state is x(t f ) = \j ■ = , the origin of state space, or it is gained by a change 

of variable. It can be shown that this would lead to a physically unrealizable state 
feedback controller, since its parameters should grow towards infinity for t—*tf. 

Therefore we will have to content ourselves with bringing the system, in final 
state, in the vicinity of the origin, i.e. in making x(/,) sufficiently small. This 

approach corresponds by the way to the physical reality, in the sense that the sys- 
tem parameters are never known exactly, and that we will never be absolutely sure 
that the system is at the state space origin in its final state, but only in its vicinity. 
This will be obtained by letting the final state free, but choosing a symmetric, 
positive semidefinite matrix S in an appropriate way, e.g. in the form of a diagonal 
matrix whose elements are much larger than those of the matrices Q and R. The 
quantities weighted by high coefficients will then necessarily become small when 
J reaches its minimum. 

The final state being now free, the transversality condition which applies here 
is equation (3.14): 

t x dh 
ax 

Since, in the present case, h W(tf) = (l/2)x T Sx , this yields, with (B.10): 

y(t f ) = -Sx(t f ). 
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Summary. The present problem is a particular case of the basic problem with 
free final state, with a cost functional given by the quadratic criterion (3.26), ac- 
companied by the following boundary conditions: 

x(0) = x , 
given t f > 0, y(t f ) = -Sx(t f ). (3.27) 

Remark 3. 7. Note that in a more general framework, including among others the 
case of time-varying systems, the Q, R and S matrices can very well depend on 
time, exactly as the matrices A, B and C describing the plant. 



3.5.2 Optimal Control Law. Riccati Equation 

From the comparison of (3.26) with (3.2), we deduce that here 

/ (x,u,0 = ^(x T Qx + u T Ru). (3.28) 

Therefore, according to (3.4), 

H (v»,x,u) = (x T Qx + u T Ru) + v|/ T (Ax + Bu) 

= x T Qx u T Ru + \i/ T Ax + \i/ T Bu . 

2 2 

Thus, with the differentiation rules (B.10) and (B.8), the following holds: 

— = -Qx + A T V . 

ax 

According to (3.13), the resulting adjoint equation is 

M/ = -A T y + Qx. (3.29) 

The value of u which makes H maximum is given by 

^ = 0, 
<9u 
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thus, according to the same differentiation rules as above, 

— = Ru + B T v,/ = 0. 

du 

The sought optimal control law, which will be denoted by u (t) , is thus: 

u(0 = u*(0 = R"'(0B T (0v(0 • (3.30) 

Remark 3.8. The matrix R~ exists, since R has been assumed positive definite, 
thus regular. 

By substitution of (3.30) into the state equation (3.25), we have 

x = Ax + BR- 1 B"V (3.31) 

Equations (3.29) and (3.31) constitute the Hamilton's equations of the system. 
Since they are linear in x and in \|/ , it is foreseeable that \|/(?) will depend linearly 

on x(t) . It is thus logic to look for a solution of the form 

y(t) = -V(t)x(t), ( 3 - 32 ) 

where P(t) is an unknown (nxn) matrix. Inserting (3.32) into the system of 
equations (3.31) and (3.29), we obtain 



and 



i = (A-BR 1 B T P)x, (3.33) 



Px Px = (A T P + Q)x. (3.34) 



The left multiplication of the first of these two equations by P followed by its ad- 
dition to the second one yields 

Px = (PA + A T P PBR "B T P + Q) x . 

Since this relation should hold whatever x, the following holds: 

-P(t) = P(t)A(t) + A T (t)P(t)-P(t)B(t)R-\t)B T (t)P(t) + Q(t) , (3.35) 

where the full time dependencies have been restored. 
The final condition Vf{t.) = — Sx(t,) applied to (3.32) has the consequence that 
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P(f/) = S . (3.36) 

The first order differential equation (3.35), which is quadratic in the unknown 
¥(t) , is called Riccati (matrix) differential equation. Since it is quadratic, it has 
several solutions P(?) . 

Any solution V(t) is symmetric: indeed, it is easy to see that, since the matri- 
ces Q(t) and R(?) are symmetric for any t, P T (?) is a solution of (3.35) if and 
only if V(t) is symmetric. We will prove in Sect. 3.5.4 that one of the solutions is 

positive semidefinite, or positive definite, according to the situation (see corre- 
sponding criteria in Appendix B). 

The solution of (3.35) is generally obtained by numerical integration, integrat- 
ing backwards in time from t = t,- with the initial condition, denoted here rightly 

a final condition, given by (3.36). 



3.5.3 State-feedback Optimal Control 

The state feedback solution to the optimal control problem of a linear system by 
quadratic criterion is obtained by combining (3.30) and (3.32): 

u*(0 = -R"'(0 B T (?) P(0 x(0 , (3.37) 

thus: 

u(t) = u*(t) = -L(t)x(t) , (3.38) 

L(0 = R _1 (0B T (0P(0 , (3-39) 

where P(t) is given by (3.35). 

The (pxn) matrix L,(t) can be calculated before starting the system. Then 
along its evolution, the control u(t) is calculated at each time from the present 
state by (3.38), which is a state feedback control. 

3.5.4 Cost Functional Minimum and Positive 
Semidefiniteness ofP 

Let us introduce the optimal control law (3.37) in the quadratic criterion expres- 
sion (3.26). Calculate first the second quadratic form: 
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u* T Ru* =( R _1 B T Px) T R( R 1 B T Px) = x T PB(R 1 ) T RR 1 B T Px, 

since P is symmetric. Since R is also symmetric, (R~ ) T = (R T )~ = R~ , and 

u* T Ru*=x T PBR _1 B T Px. (3.40) 

Let us show then that, in this case, the integrand involved in the corresponding 
cost functional (3.26), thus in its minimum value J min , has the following expres- 
sion: 

x T Q x + u* T Ru* = -—\x J (t)P(t)x(t)\ . (3.41) 

dt [ ' 

To do this, write 

— (x T Px) = x T Px + x T Px + x T Px = x T Px + x T (Px + Px). 

dt 

By using the transpose of (3.33) as well as (3.34), we obtain successively: 
d 



dt 



(x T Px) = x T (A T PBR >B T )Px x T (A T P + Q)x 

= x T (A T P PBR B T P A T P Q)x 

= x T (PBR _1 B T P)x x T Qx = u* T Ru* x T Qx , 

after taking into account (3.40). By integrating now (3.41) from t to t,- and ac- 
counting for the final condition (3.36), we get: 



x T 



(t)Y(t)x(t) = x J (t f )Sx(t f )+ f '(x T Qi + u' T Ru*)rfr. (3.42) 



By letting t = t in this expression and comparing the result with (3.26), the fol- 
lowing minimum cost functional is obtained: 



J mm =-x J (to)?(t )x(t ) 



'min— ~ A vo;* v<o; A v<o; • (3-43) 

A detailed examination of (3.42) allows to draw conclusions on the sign of the 
matrix P. The quadratic form in S is positive semidefinite by hypothesis. As to the 
integral on the right side of the relation, two situations should be considered. 
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First case. The plant to control is not completely observable. 

Its observability canonical decomposition, assuming that x 2 represents the un- 

observable part of the state, is then the following (Appendix A, Sect. A.3.3): 



V * > 

r 1 




A„ 


s 




+ 


B, 


U2, 




A2I ■"■22, 


X 2; 




B 2 




( 




y = (c, 0) 


1 










x 2 









The return to equilibrium, from an initial state x T (7) = (0 T x 2 r ) at initial time t, 

of this system equipped with a control law of the form u(r) = — Lx(t) , with 
L — (Lj 0) , thus with the system matrix 



l CL 



A BL 



'AU-B.L, 



is given by: 



x,(r) = 0, VrG^yj, since x 1 (r) = (A 11 -B 1 L i )x i (t) = 0, Vt 

x 2 (r) = exp[(A 22 - B 2 L 2 )(r - 0] x 2 , 



The control law amounts then to u(t)eO,Vt€ (,(J. If we choose now 



Q = C T C, which yields x T Qx = x T C T Cx = y T y = 0,Vt, 



since 



y(r) = C 1 x 1 (r) = 0, Vr , the integral on the right side of (3.42) vanishes. This 
means that the right side can vanish for x(t) * . The quadratic form on the left 
side is thus positive semidefinite, therefore so is matrix ¥(t) . 

Second case. The plant to control is completely observable. 

The reasoning differs from the one of the previous case as to the following 
points: the control law for the return to equilibrium u(t) is no longer identically 

zero, and the integral of the right side will be strictly positive for any t <tr , since 
it was assumed that Q is positive semidefinite and that R is positive definite. As a 
result, \ T (t)¥(t)\(t) >0 for Q<t<tf, which means that ¥(t) is in this case 
positive definite for < t < tr ■ 
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3.5.5 Solving the Riccati Equation. Hamiltonian Matrix. 



3.5.5.1 Hamiltonian System and Hamiltonian Matrix 

Let us start from the Hamilton's equations (3.31) and (3.29) of the system: 

|x = Ax + BR"B T \|/ 

[ij; = Qx- A T \|/ 

We may rewrite these equations as the homogeneous equation of an augmented 
system, which has the state vector (x T ,\|/ T ) T : 



x 

w 



A BR _1 B T ^ 



x 

w 



H 



(3.44) 



[Q -A' 

This system is called Hamiltonian linear system, of order In , and the matrix 



H 



A BR V^ 



IQ -A' , 
is the corresponding (In x 2n) Hamiltonian matrix. 



(3.45) 



3.5.5.2 Solution of the Riccati Equation 

Let <S>(t,tA be the (2«x2«) transition matrix of this system. We know that it is 

itself a solution of the homogeneous differential equation, and can thus in theory 
be calculated by solving 



with the final condition 



a>(t,t f ) = n<!>(t,t f ) ; 



<&(t f ,t f ) = \ ln . 



(3.46) 



(3.47) 



The solution of (3.44) is then given, starting from the final state MtA , by 
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x( t y 



= 0(t,t f ) 



x( tf y 
y(t f ) 
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(3.48) 



This equation can be rewritten in partitioned form as 



x(0l 



'® n (t,t f ) O n (t,t f y 
® 21 (t,t f ) <b 22 {t,t f ) 



'Mf f Y 

y(t f ) 



where the four blocks OJt,t,) are of size (nxn) . This yields 

x(t) = ® n (t,t f )x(t f ) + ® 12 (t,t f )y(t f ) 
V(t) = <S> 21 (t,t f )x(t f ) + <S> 22 (t,t f )y(t f ) 

With the use of the transversality condition (3.27), namely vf{t, ) •■ 
two previous equations become 

x(0 = \o n (t,t f )-O i2 (t,t f ) S] x(t f ) 
V(0 = \0 21 (t,t f )-0 22 (t,t f ) S] x(t f ) 

from where it results, by elimination of x(tA , that 



(3.49) 



-Sx(t f ), the 



x V (t) = [0 2l (t,t f )-(S> 22 (t,t f )S\[(!> n (t,t f )-(S> n (t,t f )S\ x(t). 

By comparison with (3.32), the solution of the Riccati equation is obtained in 
the form: 



(3.50) 



P(0 = - ® 21 (t,t f )-® 22 (t,t f )s \\o n (t,t f )-o 12 (t,t f )s 



3.5.5.3 Some Properties of the Hamiltonian Matrix 

• Pure imaginary matrix: J = 

l-i„ oj 

It is the extension of j = V— 1 to 2« dimensions. Indeed, J 2 = —l 2n , as readily 



verified. One will verify also that J 



J = J 



3.5 Linear Quadratic Control (LQC) 

• Property 1 of the Hamiltonian matrix: JHJ = H T . 

Proof. By taking into account the symmetry of Q and of R, 
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JHJ 



I 



A BR _1 B T 



lQ 



-A 1 



I 



Q A T 


' 





K 


A BR"B T 


,-K 





A T Q 
3R "B T -A, 




H T 





• Property 2 of the Hamiltonian matrix: if A is an eigenvalue of H, then —A 
is also eigenvalue of H. 

Proof. From the previous properties it follows that J~ HJ = —JHJ = — H T . The 

matrices H and — H T correspond thus to each other in a similitude transformation, 
and, on account of that, have the same eigenvalues (Appendix A, Sect. A.7.1). 
Therefore, if A is an eigenvalue of H, —A , which is trivially eigenvalue of — H 

thus of — H T , is also eigenvalue of H. 



3.5.6 Case of the Linear Time-invariant (LTI) Systems 

Assume now constant all the matrices which are involved in the problem defini- 
tion (time-invariant plant, cost functional quadratic forms without explicit time 
dependence): A, B, C, Q, R, S = constant . 
The optimal controller 

u = R _1 B T P(0 x = L(0 x 



is not time-invariant, since the Riccati equation solution P(?) still depends on 

time. 

Indeed, even though the Hamiltonian matrix is in that case constant, as shows 
its definition (3.45), the transition matrix 0>{t,tA of the corresponding Hamilto- 
nian system, which depends here only on the time interval separating the two in- 
stants, 



<S>(t,t f ) = e 



H(t-t f ) _Ht 



(3.51) 



where r = t,- —t , remains a function of time, thus also the <S>M,tA , and conse- 
quently also P according to (3.50). 
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Infinite horizon stationary solution. In order to obtain a time independent con- 
troller, thus easier to implement, one introduces the following additional hypothe- 
sis: 



t f -» oo . 

For the integral in the cost functional J to converge, it is then necessary that: 

lim x(?) = , 

from where follows the final condition: 

x(t f ) = x(oo) = 0. (3.52) 

This cancels the quadratic form in \(tj) , with associated matrix S. It disappears 
therefore from J, which reduces to: 

J = -f \x J (t)Qx(t) + u T (t)Ru(t)]dt . (3.53) 

2 Jo [ * 

The controller so designed for tf —* °° , and used instead of the optimal con- 
troller, will give to the system infinite time only an approximately optimal behav- 
ior, called suboptimal, but sufficiently close to optimal for most practical applica- 
tions. 

With the hypothesis tf — ► °° , the backward integrated solution to the Riccati 

differential equation must tend, for t — ► , towards a constant matrix. [AnMo89], 
[ZhDG96] and [DucOl], [Duc04] have shown that, for that limit to exist and to 
lead to a stable closed-loop system, it is necessary that the following conditions be 
satisfied: 

1 . the (A,B) pair is stabilizable; 

2. the (Q ,A) pair is detectable, where Q is any matrix such that Q = QjQ . 

Consequently, at the limit, P — > and the differential Riccati equation (3.35) is 
replaced by the algebraic Riccati equation (ARE) for continuous-time systems: 

PA + A T P PBR"'B T P + Q = , (3.54) 

which is a nonlinear algebraic matrix equation, having thus several solutions P, 
which are constant. 
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The unique positive semidefinite solution P = P of this equation is also the 
only stabilizing solution, i.e. the one which ensures the stability of the system fed 
back by the optimal state feedback matrix (3.39), which becomes then also a con- 
stant matrix amounting to: 



L = R"'B T P 



(3.55) 



In the cited works it is also shown that, if in addition to the previous hypotheses 
the (Q ,A) pair is completely observable, the algebraic Riccati equation has a 
unique positive definite solution, which is also the stabilizing solution. 

To solve (3.54), there are two methods. 

Direct solving. As a matter of fact, one solves the numerical equivalent of (3.35), 
which will be presented in Sect. 3.6 dealing with discrete -time systems, by look- 
ing for the stationary solution towards which P(£) converges. 

Solving by the Hamiltonian matrix. In the Hamilton's equations (3.29) and 
(3.31) of the system, x is the state vector of the closed-loop system containing the 
optimal controller, and i|/ the costate vector. 

The corresponding characteristic equation 

det(AI 2 „-H) = 



yields the eigenvalues of the Hamiltonian system. It was established in 
Sect. 3.5.5.3 that, if A is an eigenvalue, then —A is one also. If the conditions 1 
and 2 above are satisfied, there are no eigenvalues on the imaginary axis, as shown 
in [ZhDG96]. 

Let us number from 1 to n the eigenvalues 
which lie to the left of the imaginary axis in the 
complex s -plane: X l ,...,X n , and from « + l to 

2« those which lie to the right of this axis: 
A„ +1 ,...,A 2 „, as represented schematically on 

the right for an imaginary situation with n = 3 . 

The free response of the Hamiltonian system 
can be decomposed in its eigenmodes as follows 
(see equation (A.45) in Appendix A): 



/t, 
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A 5 




\ 




X 




\ 


/ 
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N' 




^ 7 

^3 
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x' 




X 




/-2 




^4 





x( t y 



In 



c t e- 



where v 



is the eigenvector associated with the eigenvalue A, . 
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Remarking that the eigenmodes associated with the eigenvalues A, J+1 ,...,A 2 „ are 
divergent, whereas it is expected that x(°°) = , it is seen that, necessarily, 
c„ +1 =... = c 2 „=0. Hence: 



x(0l 



\Vlf) 



c, e 



c„e 



c, e 



c„e 



which yields 



and, finally, 



x(0 = (v lx 



u(t) = R- l B T (y l , 



v„„ c,e 



O X (0, 



1(0 . 



(3.56) 



(3.57) 



Remark 3. 9. The eigenvalues A L1 , . . . , A Ln of the closed loop equipped with this 
optimal control are exactly the n eigenvalues of the Hamiltonian system lying at 
the left of the imaginary axis. This stems directly from (3.56), x(t) being the 
closed-loop state vector, since according to (3.44) and (3.32), 

x = Ax + BR - l B T \\i = (A BR B T P)x . 

Remark 3.10. Link with the complete modal control (Sect. 1.6). 
[Rop90] has shown that the above optimal control can be seen as a complete mo- 
dal design, in which the eigenvalues X n , ... , X Ln prescribed to the closed-loop are 
to be taken equal to the n solutions of the following equation: 



where 



det M(s) = : 



M(s) = I + R~ 1 B T (-sI-A T r 1 Q( 1 ?I-A)- 1 B, 



(3.58) 



(3.59) 



and the invariant parameter vectors are to be taken equal to the non trivial solu- 
tions of the following n homogeneous equation systems: 



M(A il .)p i =0, / = !,...,«. 



(3.60) 
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Under these conditions, the control law given by the formula (1.63) of Chap. 1 
is identical to the one which would result from the use of (3.54) and of (3.55) with 



i cost functional J — - / [x T (t) Q x(t) + u T (t) R u(f)| dt . 



One should note that, at the difference to a complete modal design of pole 
placement type, where the choice of the parameter vectors is arbitrary, these vec- 
tors are here entirely determined due to the optimality constraint imposed through 
the weighting matrices of the quadratic criterion Q and R. 



3.6 Case of the Discrete Systems 



3.6.1 Basic Problem 

The case of the discrete-time systems is very similar to the one of the continuous- 
time systems. We will thus mention here only the main differences between these 
two situations. 



3.6.1.1 Starting Hypotheses 

For the optimal control problem the given data are here: 

• a dynamic system, defined for k = 0,l,...,N — l by 

x *+i= f ( x *,u*)> (3-61) 

x given; 

• a cost functional, which involves the specified final time, t* = NT S if the dis- 
crete plant results from a sampling process at a period T s , or simply t,- = 7V if 
the time progress is counted just as a number of discrete steps: 

J = h(x N ) + ^f (x k ,u k ); (3.62) 

k=0 

• a control which remains constant during one sampling time or time slot 

u = u k = constant, for t G [kT s , (k + l)T s ] . 
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Remark 3.11. The summation limitation in (3.62) at the time step N — I expresses 
the fact that J must not involve controls out of the interval [0, NT S ] , taking into 

account that u^ is applied in the interval \N, N + 1] . 

Remark 3.12. The problem is identical to the one introduced in the continuous 
case by the equations (3.1) and (3.2). 

3.6.1.2 Modified Cost Functional 

N-l N-l 

J = /!(x A ,) + ^/ (x t ,u t ) + ^i|/J +1 [x /t+1 -f(x /t ,u /t )], (3.63) 

where the sequence {ty k } is for the time being unknown. Note here the presence 
of \\f k+1 , and not of \\f k , since this Lagrange multiplier must apply a term calcu- 
lated at k + 1 and not at k. 

3.6.1.3 Hamiltonian 

As in the continuous case, the Hamiltonian 

H(\v,x,u) = -/ (x,u) + V T • f (x,u) (3.64) 

is introduced here, its more detailed expression being 

H(\ Vk+l ,x k ,u k ) = -f (x k ,u k ) + \ V l +l -f(x k ,u k ) . (3.65) 

The modified cost functional writes thus 



r i 

J = h(x N ) + J2\\\i T k+1 -x k+1 -H(\\i k+l ,x k ,u k )\. 

<t=o 
This expression is completely similar to (3.5) and can be written also 

N-l 

J = h(x N )+J2[vI -x t - H (Vk+n x k,Uk)\ + viv - x w - Vo -x . (3.66) 
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3. 6.2 Variational Calculus 

As in the continuous case, a small variation is applied to the supposedly optimal 
control law u k , and the resulting cost functional variation is evaluated: 

8J = h(x N + bx N )-h(x N ) + y T N -d\ N — \|/J • 5x 

JV-l 

+J2[v J k 'ox* -H(\ Vk+1 ,x k +8x t ,u t + 8u k ) + H(y k+l ,x k ,u k j\ . 
A Taylor series expansion, limited to first order approximation, yields, 



£[#(l»*+i> x* +8x t ,u t + 8a k )-H(w k+1 , x k , u k )] 



k=i) 



JV-l 



: E 

/f=0 
JV-l 

: E 



^(Vt+i. x t.«t)8 x t+^(Vt+i. x t.»t + 8"t)-^(Vt+i. x *»»t) 



<9x 



9 9 

^/T(v t+1 ,x t ,u t ) 8x t +- T % w ,i t ,u t ) 8 U/fc 



9x 



9u 



Hence the expression of 8 J : 



8J: 



t/A 



c/x 

+E^-8x,-e 



bx N + yl ■ bx N - \|/J • 8x 



JV 1 



A=0 



/f=0 



9 d 

— Y H(\ Vk+1 ,x k ,u k )dx k +— T H(\ Vk+u x k ,a k )du k 
dx k du k 



or, by grouping on one hand the terms in 5x N , on the other hand those in 8x t 



8J: 



dh 



dx 

JV-l 

-£ 

k=0 



Vn - Sxa, - \|/ • 8x 



k=N \ 



Vfk 



dH^ k+l ,x k ,u k ) ] ^_^ dH(^ k+l ,x k ,u k ) ^ (36?) 



d*k J " fe^ du l k 



The objective being, as in the continuous case, to make 8 J — , the calcula- 
tion goes on as follows: 

1. one notices, as in the continuous case, that 8x = ; 
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2. y k will be chosen as solution of the adjoint equation 

d 



¥* 



dx. 



H (Vk+i^k^k) > 



(3.68) 



with the final condition, or transversality condition: 

d h 



dx h 



(3.69) 



By taking into account (3.65) and (B.16), the adjoint equation is written more 
explicitly as a difference equation: 



¥* 



<9x t <9x A . 



•Vi+i 



3. Consequently, the equation 8.7 = is reduced to: 



Af-l ^ 

8./ = ~E) t~t ^^*+i» x *' u *) 5ut = ° 

*=0 # u £ 



(3.70) 



3.6.3 Recapitulation of the Discrete Optimal Control 

Suppose that \i k and x k , k = 0,l,...,N, represent the optimal control law and 
the corresponding state trajectory, solutions of the optimal control problem de- 
fined by (3.61) and (3.62). There exists then an adjoint trajectory y k , 

k = 0,1, ... ,N , such that the following relations hold, simultaneously: 



\t+l 

d 



f ( x i> u *)> x o g iven 
df (*k,Uk) , df T (x k ,u k ) 



dx k 
dh(x N ) 
dx. 



dx t 



Vt+i 



9u, 



//(\|/,x,u) 8u t =0 



where H is the Hamiltonian (3. 64). 
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3.6.4 Case of the Discrete Linear Time-invariant Systems 

The starting data of the problem are: 

1. the discrete-time state model of a linear, time-invariant plant: 

x *+i=® x *+ ru *> x o g iven ; ( 3 - 71 ) 

2. a quadratic cost functional: 

/ = ^Sx v + ^i(xjQx t +ujRu t ), (3.72) 

h(x N ) fo( x k> a k) 

where Q, R and S satisfy the same hypotheses as in the continuous case. 

Since the final state is not submitted to constraint, the transversality condition 
yields: 



d h 

Vn = --t- 

dx k 



k=N dX N 2 



'•'' (i-xj Sx N ) = -Sx N . (3.73) 



Furthermore, according to (3.65): 

#(v*+i.**.»*) = - -( x lQ x * + «t R «t)+ vl+i (<& x a + ru t ) . 

Let us calculate dH/dx k . By successive use of (B.10) and (B.8), this expression 
becomes: 

dH „ _ T 



d*t 


KIX) 


, -i-* 


M»i+i 


and the adjoint equation writes: 








¥* = 


Q*k 


+ * T V*+i- 


Since the value of u which maximizes H 


is given by 




dH 
du k 


= 0, 





(3.74) 
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the optimal control will be solution of 

— = -Ru t +r> t+1 = 0. (3.75) 

du k 

Since a state feedback control law is expected, we will try again to find \|/ in the 
form 

V*=-P***. (3-76) 

i.e. 

Vt+i = - p i+i x t+i = - p t+i (* x * + r u k ) . 

Equation (3.75) giving the optimal control law writes then: 

Ru A r T p i+1 (Ox t + ru t ) = o, 

-(R + r T p t+1 r)u t = r T p t+1 «i>x t . 

Hence the optimal control law: 

u ; =-(R + r T p, +1 rr 1 r T p i+1 ox k , (3.77) 

which is of the desired state-feedback form: 

uI=-L A x t , (3.78) 



with: 



L t = (R + r T p, +1 rr 1 r T p i+1 o . (3.79) 



Rewriting the closed-loop state equation with this control law yields: 

x t+1 =*x t -rL t x 4 =(*-rL t )x t . (3.80) 

The substitution of (3.76) into (3.74) yields further: 

-Px A --Qx A -<D T P t+1 x /t+1 , 
which, by use of (3.80), writes now as follows: 
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-P,x,= Qx t <D T P t+1 (<I> -rL,)x, 



p t+1 (*-rL t )|x t 



Q+* T p i+1 (*-rL t )| l4 



Since this relation must be satisfied whatever x k , the following must be true: 

P /f =Q + T P t+1 (0 TL t ), 
which yields after substitution of L t by its value (3.79): 

r k = 9> T r k+1 9-9r r v k+1 r(R+r T v k+1 Tr l r T v k+1 9+Q . (3.8i) 

This equation is the Riccati difference equation. The suitable solution is a ma- 
trix P k , symmetric, positive semidefinite or positive definite according to the 
situation (see the discussion in Sect. 3.5.4). 

The final condition (3.73) yields, as in the continuous case: 

P W =S. (3.82) 

Stationary solution: Assume, as in the continuous case, that t, — > °° , thus here 

CO J 

that N — > oo . The quadratic cost functional is now J = ^ — (x^Q x k +ulKu k ) . 
The stationary equilibrium which takes then place obeys to 
P t+1 = P k = P — constant , 

and the Riccati difference equation is replaced by: 

p = o t po o T pr(R + r T pr) _1 r T PO+Q . (3.83) 

This is the algebraic Riccati equation of the discrete time systems, in short: 
discrete algebraic Riccati equation (DARE). 

P is, as previously, a symmetric matrix. Here again the suitable solution is the 
unique matrix P which is positive semidefinite or positive definite, according to 
the situation (for the existence conditions of this solution, see Sect. 3.5.6). 

The optimal state feedback matrix becomes then the constant matrix: 

L = (R+r T pry 1 r T p<i> . (3.84) 
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3.7 Optimal Control Robustness 

3. 7.1 Continuous-time Systems 

3.7.1.1 Return Difference 

Let us redraw the closed-loop system equipped with a state-feedback control law 
u = — Lx as a unity feedback system, as illustrated in Fig. 3.2. 

--G(s)=L(sI-A) _1 B ---► 



►Q— ► i = Ax + B » | \ L I t > 



Fig. 3.2 State-feedback system represented as a unity feedback loop. 

Let us start then with the algebraic Riccati equation (3.54) which corresponds 
to the stationary behavior: 

PA + A T P PBR "B T P + Q = , 

and add to it -\-sP and — sV after having changed the sign of the first side 
[AnMo89]: 

P(sI-A) + (-sI-A T )P + PBR"'B T P = Q. 
Multiply this equation left by B T ( si A T ) _1 , and right by OI-A) -1 B: 

B T (-^I-A T ) 'PB + B T P(sI-A) 1 B + B T (-sI-A T ) 'PBR VP^I-A) 'B 

= B T (-sI-A T r 1 Q(.vI-Ar 1 B. 

If we remark that, according to (3.55), RL = B T P , this equation rewrites 

B T (-5l-A T )- 1 L T R + RL(5l-A)- 1 B + B T (-5l-A T )- 1 L T RL(5l-A) _1 B 

= B T (-.?I-A T )- 1 Q(sI-A)" 1 B. 

Remarking furthermore that the first side can also be written 
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[i + B t (-5I-A t )" 1 L t ]r[i + L(5I-A)" 1 b]-R, 

we obtain finally the equality 

[i+l(-si-a) _1 b] t r[i+l(si-a) _1 b] 

= R + B t ( si A t ) _1 Q(sI A) _1 B , (3.85) 

called return difference equality [AnMo89], from the name return difference 
which has been given to the quantity I + L(sl — A) _1 B , when the plant and the 

controller are organized as shown in Fig. 3.2, and which reminds of the term 
1 + G encountered in the scalar case. 

3.7.1.2 Frequency Behavior (Nyquist Diagram) 

Let s = juj . Eq. (3.85) becomes: 

[l + L(-ywI-A)- 1 B] T R[l + L(ywI-A)- 1 B] 

= R + B T (-jwl- A T ) _1 Q (JuA- A) _1 B . 

In the SISO case, with the substitutions B = b , R = r and L = £ T , this expres- 
sion becomes: 

7'|l + £ T (ywI-A)- 1 b| 2 =r + b T (-yw-A T )- 1 Q(7w-A)- 1 b. 



Q being symmetric, we can write Q = qq T , and the previous expression becomes 

r|l + £ T (ywI-A)- 1 b| 2 =r + |q T C/'w-A)- 1 b| 2 . 



Since r > , this yields 



|l + / r (./wI-A)- 1 b| 2 >l, 



or 

ll + ^OwI-A)- 1 ^!. (3.86) 
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In order to draw conclusions on the closed-loop system stability from this ine- 
quality, we must plot the open-loop Nyquist diagram. 

In the SISO case, the open-loop transfer matrix G(s) , indicated in Fig. 3.2, be- 
comes the following transfer function: 



G(s) = G(s) 



£ T (sI 



A)-'b. 



The open-loop frequency response is thus given by: 

G(JLu) = £ T (JLul-A)- l b. 

Inequality (3.86) shows then that the distance of any point of the Nyquist dia- 
gram to the critical point — 1 will always be greater than, or equal to, 1 . In other 
words, the Nyquist diagram will never enter into the circle of unit radius, centered 
at (— 1 + y'O) , as illustrated in Fig. 3.3. 




Fig. 3.3 Example of two fictitious Nyquist plots of i T (j^ I -A) 'b (solid line and 
dash-dotted line) avoiding the critical unit disc (hatched). 



If M denotes the intersection point of this circle with the unit radius circle centered 
at O, it appears that the triangle OMN is equilateral. Thus, the smallest possible 
phase margin amounts to 60° and the gain margin is infinite. In the opposite direc- 
tion, the closed-loop stability is guaranteed as long as the open-loop gain is not 
multiplied by less than 1/2 . 
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3. 7.2 Discrete-time Systems 
3.7.2.1 Return Difference 

Let us start here with the discrete algebraic Riccati equation, (3.83): 

p = o t po o T pr(R + r T pr) _1 r T PO+Q , 



or also 



p o T PO+o T pr(R + r T pr) _1 r T PO = Q. (3.87) 



Rewrite now the first two terms of the left side, by adding and subtracting to them 
the quantity O t P z : 

P-O t PO= P <D T Pz + <D T Pz <D T PO 



= (z _1 I O t )Pz + O t P(zI - O) 



then, by adding and subtracting the quantity (z I — O t ) P O : 

P-<D T P<D 

= (z 'l-<S T )Pz-(z 'l-q> T )pq> + (z 'l-q> T )pq> + T P(zl-a>) 

(z 1 I-g> T )P(zI-a>) + (z 'l-<D T )P<D + <I> T P(zI-<I>). (3.88) 

Multiply now the two sides of (3.87) by r T (z _1 I-0 T ) _1 to the left, and by 
(zl - 0)"'r to the right, while using (3.88): 

r T pr+r T PO(zi o)~ 1 r+r T (z 1 i o T ) _1 o T pr 

+ r T (z" 1 i-o T )- | o T pr(R+r T pr)" 1 r T PO(zi-o)-'r 

= r T (z" 1 I-0 T ) _1 Q(zI-0)- I r. (3.89) 
If we use now (3.84), which can be rewritten as 

(R + r T PT)L = r T PO, (3.90) 

(3.89) can be put in the following form: 
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r T pr+(R+r T pr)L(zi o) 1 r+r T (z 1 i o T ) _1 L T (R+r T pr) 
+ r T (z _1 i o T ) _1 L T (R+r T pr)L(zi o) _1 r 



© 



r T (z 1 i-o T ) 1 Q(zi o)-'r, 



or also, after addition of R to the two sides: 

a b c d e 



[i + L(z- 1 i-o)" 1 r] T (R + r T pr)[i + L(zi-o) _1 r] 

= R + r T (z- 1 I-0 T )- 1 Q(zI-0)- 1 r . (3.91) 

The identity of these two formulae is easily proved by verifying that: 

acd = H.+ ® ace = ® 

b T cd = ® b J ce = ® 

(3.91) is very similar to (3.85) of the continuous case, where in addition to the 
conventional replacement of A by O and of B by T , one would have replaced 

-s by z _1 and R by (R + r T PT). 

3.7.2.2 Frequency Response (Nyquist Diagram) 

Let z = e JwTs , as is usual to study the frequency behavior of a discrete-time sys- 
tem, resulting from the sampling at a period T s of a continuous-time system. (3.91) 
becomes: 

[i + L(e-^' r ' i - o) _1 r] T (R + r T pr) [i + L(e y ^ i -o) _1 r] 

= r + r T (e~ Ms i o T r ' q (e JuTs i or ' r . 

In the SISO case, with the substitutions r = y , R = r and L = £ T , this ex- 
pression writes 

(r + y T Py)\l + £ T (e j " Ts I-®)~ l y\ 2 = r + y T {e~ jujTs I-0 T )~ l Q(e jujTs I-<D) _1 Y 
and, by letting as in the continuous case Q = qq T , since Q is symmetric, 
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(r + Y T PY)|l + £ T (e i ^I-0)" 1 Y| 2 =r + lq T (e y " ,r 'I-0)" 1 Y| 2 , 
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which yields 



( r + y t Py)|i + £ T (e jujTs I-0) _1 y| >r, 



or also, since r > . 



l + t(e Jwls I-®y l y\>p, 



where: 



TT+r-ypY 



<1 . 



(3.92) 



The Nyquist plot remains therefore out of a disk of radius p centered at — 1 
(Fig. 3.4). This has following consequences: 

1 . for the gain margin G M : 

lim mf G M = + />)"' 

2. for the phase margin $ M : 

sin(-$ M ) = ^> =*- *w =2arcsin^. 



Imz 




Fig. 3.4 Disk avoided by any Nyquist plot for discrete-time optimal control (open-loop). 
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3.8 Dynamic Programming. Relation to the Principle of 
the Maximum 



3.8.1 Optimality Principle 

Assume that a given optimal control problem, defined on a time interval [0, tA , 

has been solved, and call it Problem I: the solution to this problem produces some 
optimal trajectory. 

Let us define now the following Problem II: given an instant t £ [0, tA and the 

corresponding point x(f) of the optimal trajectory determined above, find the so- 
lution of the new problem of optimal control starting from this point, which is 
taken as new initial state. 

The optimality principle claims that the solution to Problem II corresponds ex- 
actly to the rest of the solution to Problem I, from this point on. Let us reformulate 
this principle, stated by Bellman: 

Optimality Principle. Starting from any arbitrary point of an optimal trajectory, 
the remaining part of the trajectory is optimal for the optimization problem corre- 
sponding to this point taken as new initial state. 



3.8.2 Continuous-time Systems. Hamilton- J acobi- 
Bellman Equation 



Hypotheses: 



i = f[x(0,u(0], 

x (°) = x o given, 

u(0 e «U , 



h[x(t f )} + J^' f [x(t),u(t)]dt ■ 



Let F[x(7),?] denote the optimal (minimum) cost functional associated with the 
motion from some state x(t) until the final state x(tj ) : 
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V[x(t),t]=V[x,t] 

= mm\h[x(t f )} + J'' f [x(T)Mr)]dT\ . (3.93) 

The present goal is to find an equation for V \x, t] . 

We start with the hypothesis that Fis known for t + 8t , where 8 Ms a small 

time interval, and proceed backwards towards time t. By application of the opti- 
mally principle, 

F(x,0 = min J f [x( T ),u(r)]dT + V(x + 5x,t + 5t) , (3.94) 

where the integration interval [t, t f ] has been split into [t, t + dt] and [t + 8t, t f ] . 

If we suppose that during the first interval the applied control, u(t) , remains 
practically constant, we can write: 

n t+St 

J /o [x(t),u(t)]Jt ~ f [x(0,u(0] 5t . 

Moreover, by first order Taylor series approximation, 

V(x + bx,t + 50 ~ V(x, t) + dV( - x '^ & 

dt 



dV(x,t) dV(x,t) dx 



dt dx 1 dt 



St. 



^V(x,t) + 

By substitution in (3.94), this yields: 

V(x,t) = n\m\f Q (x,\\)ht + V{x,t)+ "' y " ,l ' bt+ "' v t. ,v -f(x,u)8? 



[., , 5 „, s dV(x,t)r. dV(x,t) 

__t / (i,u)&+f(i,o+ ; &+— -^t ■ 

u [ at dx 



Since V(x,t) does not depend on u, this term can be extracted from the second 
member brace, and then subtracted from both sides: 

= mm[f (x,»)5t+ dV(x,t h t+ dV(x ; t) -f(x,u)5t 
« [ dt dx 

The second term of the sum in the brace does not depend either on u, and can thus 
also be withdrawn from the minimization. By dividing further the two sides by 
5t , we obtain finally: 
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0= ; ,; + min / (x,u)+ vw -f(x,u) . (3.95) 

at u [ g x ' j 

This equation is called Hamilton- J acobi-Bellman equation. Its numerical solu- 
tion is calculated backwards, by starting with the evident final condition according 
to (3.93): 

V(x,t f ) = h[x(t,)] . (3.96) 



3.8.3 Relation to the Maximum Principle 

Letting the costate vector of the previous method be 

y = -^>, (3.97, 

ax 

the Hamiltonian of (3.4) is retrieved in the form 

H = -f (x,»)-^fi-f(x,») , (3.98) 

ax 

and (3.95) can be written: 



= ^-^ + mm \-H\ . 

dt « l J 



Remarking that min \—H\ = — max [HI , we obtain the following equation: 



8V{x,t) _ r g i (399) 

dt u L J 

The Pontryagin's maximum principle has been regained here. The optimal con- 
trol u is the one which maximizes the Hamiltonian. 
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3.8.4 Example: the LQC Problem 



Consider the linear system x = A x + B u , and the quadratic criterion 
I f' f 

made. 

Let us look for a solution of the Hamilton- Jacobi-Bellman equation of the form 



1 C*f T T 

J = — I (x Qx + u Ru)d, where the simplifying assumption S = has been 

2 Jo 



F(x,0 = -x T P(0x, (3.100) 

where P(7) is an (nxn) , symmetric, unknown matrix. 

The partial derivative of V(\,t) with respect to x, thus at t constant and, hence, 
at P(t) constant, amounts, according to (B.10), to 

^> = F(0*. 
ax 

from where it results by transposition that 

M^) = x T pTw = xTpWj 

OX 

so that the Hamiltonian of this system writes, according to (3.98): 

H = -(x T Qx + u T Ru) x T P(0 (Ax + Bu) . 

Let us determine for which value of u this expression is maximum. According to 
(B.10)and(B.8), 

dH r t i T t 

— = Ru x T P(0B = Ru B T P(0x. 

da [ > 

The maximum of// will thus be obtained for 

u= R _1 B T P(0x, 

and will thus have the following value: 
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i/ max =--[x T Qx + x T P(f)B(R" 1 ) T RR" 1 B T P(Ox]-x T P(f)[Ax-BR" 1 B T P(Ox]. 

R being symmetric, (R~') T = R _1 , and 

H max = - 1 x T Q x + 1 x T P(0BR" 'B T P(0x - x T P(0Ax , 
or, with 

x T P(0Ax = -x T [P(?)A + A T P(0]x , 

H m!a = - - 2 x T [Q - P(0BR" 'B T P(0 + P(0 A + A T P(0] x . 
Equation (3.99) becomes thus: 

Q _ _ d F(x, _ 1 x t Tq _ p(^BR-'B T P(0 + P(f ) A + A T P(Ol x . 

dt 2 [ * 

The partial derivative of V(x,t) with respect to t, thus at x constant, writes: 



dV(x,t)_ d 



-x T P(0x =-x T P(()x. 
2 2 



dt dt 
The Hamilton-Jacobi-Bellman equation is then: 

= -x T [P(0 + Q + P(?)A + A T P(?) - P(0BR _1 B T P(0]x . 

For that expression to be identically true, i.e. whatever x, P(t) must be chosen so- 
lution of the equation 

- P(0 = P(f)A + A T P(?) - P(0BR _1 B T P(0 + Q , 

which is nothing else than the Riccati differential equation (3.35) found in 
Sect. 3.5.2. 

Remark 3.13. Since, according to (3.93), J mia = V(\,0) , we deduce from what 
precedes that 
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^ ml „=^x T (0)P(0)x(0) • (3-101) 

We have thus regained also the expression (3.43) of the cost functional corre- 
sponding to the optimal control law of Problem I. 



3.9 Solved Exercises 

Exercise 3.1 Optimization of a Mechanical System 

Consider the continuous-time linear system described by the state equations 



K 

— u 



These equations describe the operation of a pair of scissors aimed at cutting on the 
fly a paper band moving at constant velocity. The state variables x x {t) and x 2 (t) 
are respectively the circular motion and the tangential speed of one of the two 
scissor blades, K and r m are the proportionality constant between speed and ar- 
mature voltage of the motor driving the blade and its mechanical time constant. 
Since the cost of motors increases with their power, the goal of the present exer- 
cise consists in optimizing the control of this device so that the mean effective 
torque of the motor is minimized between two consecutive cuts, i.e. between the 
times and tt , which will determine the minimum motor power required. This 

amounts to minimizing a term proportional to the integral of the square of the 
blade angular acceleration, or tangential acceleration, on the considered time in- 
terval. The resulting minimization criterion will thus be 



T i r' r -2 j i ("'{ i 

J = -\ x 2 dt = -\ x 2 



^2 
K 
— u 



dt . 



The final state x(tA is of course entirely determined by the necessity for the 

two blades to be aligned against each other at the time?, of the next cut and for 

their tangential speed to match then that of the linear motion of the paper band. To 
simplify the problem, it is not asked to introduce these requirements in the calcula- 
tion. 
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Although the previous criterion is quadratic, it is not the usual LQC design cri- 
terion and the solving of this problem will not rely on the Riccati equation. Adopt 
instead the following approach: 

a) Write the Hamiltonian expression and deduce from it the adjoint state 
equation and the optimal control law, expressed as a function of the costate. 

b) Substitute this law into the state equation and the adjoint state equation. 

c) Calculate the general solution of the obtained differential equations, by in- 
troducing as many arbitrary constants as required ( q , c 2 , ■ ■ ■ ), and deduce 

from this the form of the optimal control law expressed as a function of 
these constants and of the model parameters. 

Solution: 

(a) With the notations of Sect. 3.2, equation (3.2) and fallowings, it holds that 



f (x,u,t) = 



1 



K 



Z = f(x,u,t) 



K 



According to (3.4), with \|/ T = (tp t ip 2 ) ■> the Hamiltonian is thus 



H = — 



K 



+ ^1*2 +V , 2 



K 



which yields the adjoint state equation (3.13), componentwise: 



/ dH n 



i' 2 



dH 

dx 



K 



T... 



To obtain the optimal control law u , maximize H with respect to u: 
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dH _ K 

du T 



K » 



+ —^2=0, 



K K 



ip 2 



(b) By substitution, 



K * 1 1 , 

U = X 2 H X 2 +1/7 2 =f/J 2 

T„. T„. T_ 



^i=0 



V> 2 =- 



1 1 

x 2 H x 2 + f/) 2 



H-Vi "02 



:-^ 



(c) The general solution of these differential equations is therefore: 

tl) i = Cj (constant) 

^2 — — I V'i (t)</t = — q? + c 2 

r t 2 

x 2 = J ip 2 ( T )dT = — C; hc 2 l + c 3 

3 2 

Xj = / x 2 (r)fi?T = — Cj — 
The optimal control law is thus finally: 



u U) = —t 

2K 



c \ ,2 | C 2 ~ C l 



Ct — CiT_ C 3 +C 2 T 



a: 



-*+- 



a: 



where the constants c x , c 2 and c 3 will be determined by the conditions imposed 

to the final state. The result is that the best control imposes to the scissors a time 
behavior which is a parabolic function, obtained by a control law itself parabolic. 



Exercise 3.2 Optimal Control of the Double Integrator 



Consider the following system described in state space by: 
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Xi — .l"-j 



a) Calculate the optimal control law in stationary state which minimizes the 
cost functional 



1 c°° 

J = - I ( / o 2 x, 2 +u 2 ) dt, with p > . 

2 Jo 



b) Determine the corresponding poles of the closed-loop system and comment 
its stability as a function of p . 

Solution: 

(a) The state equation of this system is of the form 



x = Ax + b u , A- 



o r 


,b = 


'0" 







1 



and the cost functional is the quadratic criterion 



J = - f(x T Qx + u T Ru)dt, Q 

2 Jo 



P 1 o 




R = l 



Let us write the algebraic Riccati equation (3.54), which corresponds to the sta- 
tionary case. We have, successively 



B T P = 


= (0 1; 


Pn Pn 

,Pl2 Pll, 


= {pn 


Pii), R _I =i, 














/ 

PBR _1 B T P = (B T P) T (B T P) = 


Pn 


{pn Pn) = 


Pn 


P12P22 

2 


3 


.-P22J {P11P12 


/>22 J 


' Pn Pn 
,Pn P11, 


r 



+ 


o s 

1 


P 
,p 


11 Pll 

12 -P22 


- 


P11 PnP. 

1 
P11P21 Pn 


.2 


+ 


( 2 
p 




0' 

0, 


/ 


o s 





This matrix equation is equivalent to the following system of three nonlinear equa- 
tions: 
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2 i 2 n 

-Pn+P =0 
Pu-PnP72= Q 

2 Pn-P22=° 



Due to its quadratic nature, this system has more than one solution. It is easy to 
verify that the positive definite solution is given by: 



p^lp p 
p Jlp 



2p 2 — p 2 = p 2 , are all strictly positive (see Ap- 



since the determinants associated with the upper left submatrices of P, 

r— P Jlp~ p 

p n =pyJ2p and — = 

p 4 2 p 

pendix B, Sect. B.6). 

The resulting control law is then: 



M = -R" 1 B T Px=-fo r 



p^lp p 
P -ftp, 



^2p)x=-i 1 x. 



(b) The closed-loop system matrix is: 



A a =A-W' 



(0 I) 




(o) 



(p Jz~p\ 



(0 1 ) 



-p -\I 2 P, 



and the closed-loop characteristic polynomial is given by: 



det(sI-A CI ) 



s -1 
P s + Jlp 



■■ s{s + y]2p) + p = s 2 + yJ2p s - 



The closed-loop poles are thus s = sJp/2 (— 1 ± j) , which characterizes a second 

order system with an undamped frequency 0J n = -Jp and a damping coefficient 

( = l/v2 . Its stability is therefore guaranteed for any value of p , and its phase 
margin is superior to 60° (Sect. 3.7.1.2). 
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Exercise 3.3 Optimal Control of the Three-tank System 

We consider here again the three-tank system introduced in Exercise 1.2. Design 
here an optimal control which brings the plant back to equilibrium after a distur- 
bance of its initial state given by x =(—0.8 —0.5 —0.2) , value already used 

in that exercise, in order to compare the results. Take for the two weighting matri- 
ces of the quadratic criterion the identity matrix in a first trial, and then another 
matrix R aiming at reducing the initial control. 

Solution: 



Run the program MMCE.m for this plant and choose the synthesis of a state 
feedback by the quadratic criterion (LQC) algorithm with state weighting. 
With Q = I 3 , R = I 2 , the following control law is obtained: 



f0.776 0.331 0.1061 
0.106 0.283 0.620 



M: 



(1.159 0.054 
0.030 1.230 



The responses of the measured state variables y l = x l and y 2 = x 3 , and the 

corresponding control signals, are reproduced in the left column of Fig. 3.5. Since 
one of the control signals of this trial comes close to the maximum value of the ac- 
tuators (1 m 3 / mn ), make a new trial with the choice R = 10 x I 2 , which yields 



1.181 0.107 0.056) 
0.056 0.076 0.110 



M: 



0.452 
-0.123 



-0.107 
0.617 



The responses of the same state variables and the associated controls are repro- 
duced in the right column of Fig. 3.5. 

The comparison with the plots of Exercise 1.2 shows that the dynamic re- 
sponses are almost the same, especially in the first of the two cases studied here. 
In the second studied case, the controls are about four times weaker, but the re- 
sponse has become slower. 

The advantage of the LQC control, as compared with the eigenvalue placement 
techniques, is the simplicity of the design. The drawback is that we have no con- 
trol anymore over the dynamical behavior of the closed-loop, contrarily to the pole 
placement or modal control methods. This behavior will have to be adjusted in the 
case of quadratic control, by repetitive trials and simulations. 

In Exercise 6.1 of Chap. 6, another design method will be described, which al- 
lows an LQC synthesis with regional pole constraint. 
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Case Q = I 3 , R = I 2 



CaseQ = I 3 ,R=10xI 2 



o 

-0.1 
-0.2 
-0.3 

-0.4 
-0.5 
-0.6 
-0.7 

-0.8 
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t (mn) 
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Fig. 3.5 Free response of the state variables X\ and X3 (top) and associated controls (bottom). 



Exercise 3.4 Optimal Control of the Inverted Pendulum 

This exercise deals again with the inverted pendulum LIP 100 (Amira), but this 
time we will design an optimal control for it. 

a) Design an LQC optimal control, by choosing for the cost functional 

1 f °° T 2 

J = — | (x Qx + Ru ) dt successively the weighting matrices 

2 Jo 

Q = 100xl 4 , R = \, and then Q = I 4 , i? = 100. 



b) Verify by simulation that these two control laws stabilize the pendulum and 
compare the control signal in the two cases. 
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(a) The LQC controller and gain compensation matrix are, respectively: 
Case(l): Q = 100xl 4 ; R = \: 



-10 34.28 32.37 12.40 ; M ■■ 



10. 



Case (2): Q = I 4 ; R = 100 : 

£ T =(-0.1 0.8686 0.8370 0.2119); M = -0.1 . 

The step responses for a 1 volt position reference step applied at t = 1 s are given 
in Fig. 3.6 in the two cases. 



1.2 
1 
0.8 
0.6 
0.4 
0.2 
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Case (2) 
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v,> 




-/--—-■-- 



4 6 

r(s) 



Fig. 3.6 Closed-loop step responses for the two LQC controllers. 

(b) The control reaches —10 volts in case (1) and —0.1 volt in case (2). The 
choice of the weighting matrices favors significantly the energy savings in this 
second case, to the detriment of response speed. 



Exercise 3.5 Optimal Acceleration of a Shopping Cart 



Calculate what should be the optimal variation with time of the force u(t) to be 
applied to a shopping cart, in the following conditions and hypotheses: 

• the cart mass is taken as mass unity; 

• the cart moves along a straight line, considered as the Ox axis, and is submitted 
to a viscous friction force — kx(t) , where x(t) is the cart position at time t ; the 
cart is initially at stop, at the origin of the Ox axis. 
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• the objective is to maximize the distance covered by the cart in a specified time 
tf , while minimizing the total effort. 

Solution: 

This one -dimensional problem is described by the following motion equation: 
x(t) + kx(t) = u(t), x(0) = 0, x(0) = 0. 
The cost functional taking into account the optimization requirements is: 

'/ 1 



r i 1 
J = -x(t f )+ I -u 2 (t) 

1 J o 2 



dt . 



A minimum value of J will thus correspond to a maximum value of x{tA and to a 

minimum of effort spent all along the trajectory, as much for accelerating as for 
braking the cart. 

With the state variables x 1 = x and x 2 — x , the system state representation is: 



x = 






1 ' 


x + 


V 


.0 


-k J 




1 



u, jc!(0) = jc 2 (0) = : 



(3.102) 



and the cost functional becomes: 



r '/ 1 

J = -x l (t f )+ I -u 2 {t)dt. 
J o 2 



This is a free final state, fixed final time problem, the basic problem of optimal 
control. With the notations of (3.2), 



h[x(t f )] = -x,(t f ) 



By rewriting (3.102) in the form x = f (x,w) , the components of fare identified: 



f = 



— kx 2 + u 



We calculate then the Hamiltonian, according to (3.4): 
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v T, f = (V'i tpi) 



/vAi — r" U 



= 1piX 2 —klp 2 X 2 +1p2 U ' 



1 



H = u + tp x x 2 + ip 2 u — k tp 2 x 2 . 



The optimal control law will be given by 
dH 



du 



= o 



u + ip 2 = => u (t) = ip 2 (t) . 



From the Hamiltonian expression the adjoint state equation is derived by (3.13): 



4\(.t) = —— = 

ox. 



lp 2 (?) = — — = -^ + k 1p 2 

ax n 



With the transversality condition (3. 14), 

dh 



W f ) 



dx 






the solution for the first component of the costate vector is ij\(jt) = constant = 1 , 
which yields for the second one 

ip 2 (t)-kip 2 (t) = -^(t) = -l. 

Since the expected solution has the form tp 2 (t) = Ae + C and the final condition 

is ip 2 (tj) = , we obtain 



A(t) = - 



1 



l-e 



-k(tr-t)\ 



Hence the optimal control law: 



u (t) = 



1 



l-e 



-k(t f -t)\ 
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This law has a falling exponential shape, as illustrated in Fig. 3.7, which is plotted 
in arbitrary units and has been obtained for tr = 10 and k = 0.4 . 



u\t) 




tf = 10 



Fig. 3.7 Shape of the optimal control law to move the shopping cart. 



Exercise 3. 6 Discrete-time Optimal Control of an Air- 
Flow Heater 



The plant used here is the "Process Trainer PT326" setup of the company Feed- 
back™. It is an example of noisy system, and will be reused on grounds of that in 
the exercises of Chap. 4. 

This process operates as a hair dryer: air is driven by a centrifugal blower 
through a tube, at the input of which it crosses a heater grid. The air temperature y, 
which represents the signal to be controlled, is measured by means of a thermistor 
inserted into the air stream at several points along the tube, here at its output. The 
control signal u is the voltage applied to the heater grid. 

This plant is available in the MMCE.m program under the name process 
trainer PT326. Its continuous-time transfer function is (in rounded values): 



-0.2s 



G(S): 



(1 + 0.25 s) 



2 ' 



By sampling with zero-order hold at T s = 0.2 second, the following discrete 
model is obtained, by the statement c2dm applied to the observability canonical 
model 2 : 



2 The state variables numbering of MATLAB® in the canonical forms is the inverse of that used 
in this book. This explains, among other, that y k = Xj ■ and not X 2 % ■ 



H+l 



' 0.0899 0.0899) 
-1.4379 
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(0.1912) 

2.9675 



In this exercise, the plant is to be controlled by optimal control. 

a) Calculate the discrete-time optimal control law which minimizes the fol- 
lowing quadratic criterion: J = ^ (x^Q \ k + Ru\) , by taking Q = I 2 

k = 

and R = 100 . 

b) Repeat this design with other ratios between the two weighting matrices, 
such as Q = 100xl 2 ; R = 1 . Observe by simulation the effect of these 

choices on the step responses of the output and on the control signal. 

c) Verify the robustness of this control law by using the approach in 
Sect. 3.7.2 and by plotting the Nyquist locus of the compensated open loop, 
for Q = I 2 ; R = 10 . 

Solution: 

(a) Run the MMCE.m program by choosing successively the plant to study #3, 
then discrete model, synthesis of state feedback, without integral action, 
quadratic criterion algorithm (LQC) with state weighting, and introduce the 
values of the exercise statement. 

The following control law is obtained: 

£ J =(-0.0691 0.0335); Af = 1.1993. 

The step response of the measured output y = x x and the control signal u are 
recorded by simulation and plotted on the left side of Fig. 3.8. 

(b) The control law is now: 

£ T = (-0.4796 0.2732); M = 2.7057, 

and the output step response and control signal are reproduced on the right side of 
Fig. 3.8. It is seen that the transient response is faster (about 0.5 s rise time, versus 
1 s in the first case), but at the cost of a much more energetic control. 

This is in agreement with the chosen weightings: in this second case, indeed, 
we have required small trajectory deviations (large coefficients in Q), while we 
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did not require control energy minimization (small R). The first case is the exact 
opposite of this one. 



Case Q = I, R = 100 



Case Q = 100 x I, R = 1 
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Fig. 3.8 Step responses and corresponding control signals for questions (a) and (b). 

(c) The closed-loop poles are 0.4062±y'0.1049 in the first case, and 0.1785 
and 0.0013 in the second one. 

The closed-loop system is thus stable. To check its robustness, plot with the 
script file LQ_LQG_robustness_discrete.m, contained in the downloadable 
mmce.zip archive, the Nyquist diagram of the compensated open loop and the disk 
which it should avoid, according to (3.92) in Sect. 3.7.2.2. As shown in Fig. 3.9, 
this is indeed what happens. 
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Fig. 3.9 Compensated open-loop Nyquist diagram (dashed line), forbidden disk 
(hatched), and unit circle (dash-dotted line). 



Exercise 3. 7 Optimal Control by Solving the ARE. 



Consider the plant 



U(s) 


r \ 
1 

s{\-s) 

\ A 


1 Y(s l 




1 




'*mmaaa^^^. 





a) Determine the state space representation of this plant in diagonal form. 

b) Calculate the state-feedback optimal control law, which minimizes the cri- 

1 f°° 2 2 

terion J ~— \ (y + u )dt . 

2 Jo 

c) Study the robustness of this control law by following the approach of 

Sect. 3.7.1. 



Solution: 



(a) By partial fraction expansion, the above transfer function can be decom- 
posed as 



1 1 1 



Y(s) = = 

U(s) s(l-s) s s-l 
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X = 


o o N 

1, 


x + 


r 
i 


y = 


(1 - 


l)x 





(b) According to (3.28) we have 



/„ = -(y 2 +u 2 ) = -(x J C J Cx + uRu) = -(x J Qx + uRu) , 



where 



Q = C'C = 



r 


(i 


-i)= 


i 


-f 


-1, 


-l 


i ; 



; R = i 



The algebraic Riccati equation (3.54) is thus, with P (symmetric) = 


Pu 

P12 


Pn 
P22, 




p u N 


+ 


' N 




Pu Pn 


\ 1 


Pu Pu 




-1 r 






P22, 




Pl2 Pll, 




Pl2 P22, 


1 1 


,Pl2 P22, 




1 


-\ ; 





It splits into the following three equations: 



-Pu(Pu+ Pn)- Pn(Pn+ Pn) = ~ l 
Pn-(Pu+Pn )(Pn +P22) = l 
2p 2 2 ~(Pn+ P22 )(Pn + P22 ) = - ! 



By substitution and elimination, the positive definite solution of this system of 
quadratic equations is determined: 



V3 -1-^ 
1-V3 3 + 2V3 



The optimal controller in stationary state is thus: 



L = £ J =i? _1 b T P = (l f 



V3 -1-V3 S 
1-V3 3 + 2V3 



= (-1 2 + V3 
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N.B.: this solution can also be obtained numerically by the MATLAB® state- 
ment lqry(A,B,C,0,Q,R), which uses directly a quadratic form of the output, 

y T Qy 

(c) The Nyquist locus of the compensated open loop, plotted by means of the 
script file LQ_robustness _continuous.m available in the downloadable mmce.zip 
archive, is shown in Fig. 3.10. 




Fig. 3.10 Nyquist diagram of the compensated open loop (continuous case). 



As expected, the Nyquist locus does not enter the unit radius disk centered at 
— 1 . This guarantees the closed loop an infinite gain margin and a phase margin of 
at least 60° (see Sect. 3.7.1.2). 



4 Noisy Systems - Optimal Linear Filtering 



This chapter deals with noise signals, also called random or stochastic processes, 
which affect the plant itself or the available measurements. As mentioned earlier, 
the state estimator will be here a filter. The optimal linear filter, if optimization is 
understood as minimization of the estimation error variance, is the well known 
Kalman filter. This chapter is organized as follows: after an introduction to the es- 
timation theory, the description of noisy systems in state space will be presented. 
The obtained equations will then be used to establish the Kalman filter, first for 
the discrete case, which is easier to apprehend in an initial approach than the con- 
tinuous case, mainly for notation reasons, and then also for the continuous case. 
The choice of the noise covariance matrices will be discussed after that, and the 
use of a Kalman filter as an observer in the control loop will be presented. Though 
this book is entirely devoted to linear systems, a short digression will occur about 
an extended version of the Kalman filter, which applies to nonlinear systems. 

All over this chapter the duality with the optimal control approach of Chap. 3 
will be emphasized and lead to the Duality Principle, which reaches far beyond 
the questions discussed here. 

The basic notions of Probability Theory, for scalar or vector signals, and the 
notations used in the present chapter are summarized in Appendix C. More details 
can be found in textbooks about this topic such as [Kay06] or [ShBr88]. 



4.1 General Theory of Estimation 

The problem of estimating the state of a noisy system, thus of estimating a random 
vector, will be cast first in the context of the estimation theory. 



4.1.1 Introduction to the Estimation Theory 

It is useful here to recall a few basic notions, in order to highlight the differences 
between the deterministic and the stochastic situations. 
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4.1.1.1 Unknowns, Measurement, Estimation 

Two categories of variables will be considered in the sequel, the word variable be- 
ing used in the large sense which encompasses as well scalar as vector variables: 

x: the unknown variables to estimate 

y: the measured variables, whose value depends in some way on that of x. 

• x can be: 

- either the vector of parameters of a system to be identified: x = 9 ; one 
speaks then of system identification; e.g. if a SISO discrete-time system is 
to be identified in terms of a z transfer function, 

rl , b +b lZ - l + ... +b„z~" 



l + a,z 



the vector of parameters will be: 6 = (a, a 2 ■■■ a n b b ■■■ b t 



\T 



or the set of successive states of a system, e.g. in case of a discrete-time 
system, the extended vector x = x, x 2 ••• x k ••• , or simply the 



system state vector x = x k at time t — kT. In this case one will speak of 
state estimation, or reconstruction. 

• y will be the set of made measurements, e.g. in the case of a discrete-time sys- 
tem the input-output measurement 

T / T T T 

y =(yi y 2 ••• y* 

In general, the measurements y depend on the value of x. This dependency is 

- deterministic, if y is completely determined by x; 

- stochastic, if y depends not only on x, but also on chance due to noise or 
unknown disturbances. 

In summary, the situation can be sketched as follows: 

deterministic or stochastic 

denendence 
unknowns x - > measurements y. 

To estimate x as a function of y consists in manipulating these measurements to 
obtain a quantity x called estimation, or estimate, of x: 

deterministic dependence . - 

measurements y > estimate x 
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The considered manipulation is deterministic in the sense that, to a given vector 
y, corresponds a deterministic value of x . An estimate is thus a function of y: 

x = g(y)- 

There are many identification or estimation methods, thus many estimation 
functions g(y) , which will yield different estimations x , more or less satisfac- 
tory, the degree of satisfaction depending of course on the closeness of x to x. 

To appreciate whether a given function g is satisfactory, it will be necessary to 
adopt a statistical viewpoint, and to consider that part or all of the variables (x, y) 
involved in the problem are realizations of random variables. 



4.1.1.2 Variables to Estimate and Measurements, as Realizations 
of Random Variables 

Le us consider indeed the experiment which consists in recording the inputs and 
the outputs of a given plant type, for its identification. One such record is 

y T =(uo u^ ••• u^ yj y^ ••• y^ ). By repeating the experiment, a dif- 



ferent vector y will be obtained, for one reason because of measurement noise, but 
also because the input sequence may well be random from one test to the next one, 
as is the case in system identification by pseudo-random binary sequences: 

• one will therefore say that y is a realization of a random vector Y; 

• consequently, given some estimation function g, x = g(y) will be a realization 
of a random vector X = g(Y). 

Remark 4.1. As is common practice in Probability Theory, lower case letters de- 
note the realizations, and capital letters the random quantities (see also Notational 
Conventions, at the beginning of this book). 

The random variable X is called estimator. The realization x is called estima- 
tion or estimate. 

As far as the true vector x (parameters, unknown states) is concerned, two 
situations can be envisioned: 

• 1 st case: x is a realization of a random variable X. The corresponding theory is 
called general estimation theory. 

This is the case, e.g., when one wants to estimate the parameters of a class of 
systems or to estimate the state of a given system, which is corrupted with 
process noise: if the experiment were repeated, x would present each time a dif- 
ferent value. 
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This is the situation which will prevail for the remainder of this chapter, and it 
is thus the general estimation theory which will be developed here. 

• 2 nd case: x is an unknown deterministic quantity. The relevant theory is called 

classical estimation theory. 

This is the case of the identification of a given system, where all successive 
measurement data y are recorded on the same system with the same state vector 
x, or of the estimation of the state of a system immune to process noise. 



4.1.1.3 Estimation Error 

In the two previous cases, the estimation error will be defined as the difference be- 
tween real value and its estimate: x = x x . This error, unknown if x is un- 
known, is always a realization of a random variable X . Indeed, according to the 
situation envisioned, the following will hold: 

X = X — X (general theory), 
X = x — X (classical theory). 

To evaluate the quality of a given estimation function g, EJXi is often used, 

i.e. the mathematical expectation of estimation errors which would be observed in 
a large number of experiments, growing at the limit towards infinity. 



4.1.2 Linear, Unbiased, Minimum Error-variance 
Estimate 

Consider two random vectors, X of length n and Y of length q. 

4.1.2.1 Linear Estimation 

One calls linear estimation of X from Y an estimation x = g(y) for which g is a 
linear function of y: 

x = Ky + l . (4.1) 

x is then a realization of the linear estimator 
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X=KY+^ . (4.2) 

4.1.2.2 Unbiased Linear Estimation 

Let X = X — X be the estimation error. An estimation is said unbiased, if 

E{x} = . (4.3) 

The linear estimate (4.1) will thus be unbiased if 

EJx} = EJX-x} = E{X-KY-.£} = |A x -K|j, r -.£ = 0, 

where \i x = E {X} , p, r = E { Y} , i.e. if £ = \l x — K \l y . The unbiased linear es- 
timate of X from Y is thus 

x = l i x +K(y- l i Y ) , (4.4) 

corresponding, of course, to the estimator 

X=n x +K(Y-M • (4.5) 

4.1.2.3 Optimal (Unbiased) Linear Estimate 

Let us look now, in the class of unbiased linear estimations, for an estimation 
which minimizes the estimation error variance. 

Since Ei X [ = , the estimation error variance is by definition the covariance 
matrix 

E = Var{x} = E{xX T } = E{(X-X)(X-X) T }. 
According to (4.5), 

E = E{[(X-^)-K(Y-^)][(X-^)-K(Y-^ y )] T 
= E{(X-^)(X-^) T }-E{(X-^)(Y-j J L r ) T }K T 

-KE{(Y-^)(X-^) T } + KE{(Y-^)(Y-^) T }K T , 
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thus, using the notations of (C.9) and (C.l 1) in Appendix C, 

£ — ^ XX ^JiT K £j>y + K 2jyyK . (4-6) 

The problem consists in finding the (n x q) matrix K which minimizes this er- 
ror variance S . As a matter of fact, since we do not know how to minimize a ma- 
trix, and owing to the particular structure of S (see (C.9)), we will look for the 
matrix K which minimizes the sum of the estimation error variances of all the 
components of X, i.e. the following criterion: 

■/ = £>*{*,} = £e{*, 2 } = e{x t x}. (4.7) 

By application of (B.2) of Appendix B, 

E{x T x} = E{tr[xX T ]| = trfE{xX T }l, 

so that, finally: 

7 = tr(E). 

According to (4.6), 

tr(E) = tr(Z x ,)-tr(Z^K T )-tr(KZ^) + tr(KZ yy K T ). 

Since, according to (C.12) of Appendix C, E 1 ^ = 2. XY , the third term on the right 
side of this equation evaluates as 



Hence: 



tr(KZ y;r )= tr(KI yx ) T = tr(Z yx K T ) = tr(Z xy K T ) . 



tr(Z) = tr(Z x ,)-2tr(Z xy K T )+tr(KZ yy K T ). 



By applying the differentiation relations of a scalar with respect to a matrix, 
(B.19) and (B.21) of Appendix B, we obtain 

-^-tr(S) = -25V+2K5V, 



since Z yy is symmetric (see (CIO), Appendix C). The minimum of tr(Z) is ob- 
tained by cancelling this derivative, i.e. for K solution of 
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yielding 



JV Zlyy ^XY ^ ! 



K = -L XY -L Y - Y 1 . (4.8) 



Substituting now this value in (4.6), we obtain for Z : 

^ = ^xy — **■ ^k? = ^xx ~ ^jit ^yy ^nf • 

Let us gather the previous results together. 

Theorem 4.1. If X a«c/ Y are two random vectors, with variances and covari- 
ance E^ , ~L YY and ~L XY , the unbiased, linear estimate of X from Y , which is 

optimal in the sense of the minimum variance of X = X — X , is defined by 

\ = ii x +I. XY 'L Y ^\y-ii Y ) , (4.9) 

estimate for which 

Y. = Vn{x\ = Y. xx -Y. XY ?.-% x . (4.10) 

Remark 4.2. The optimal linear estimator corresponding to (4.9) is given by 

X = i L x + X XY X YY -\Y- i , Y ) ■ (4.11) 

Remark 4.3. By developing (4.7), 



j=±^}=A±i 



2 



= E £(X,.-X,) 2 



it appears that the above minimization amounts to minimizing the sum of the 

squares of the differences between the various components of X and of X , in sto- 
chastic average (average norm of the error vector). This is why the linear optimal 
estimate obtained above is also said optimal in the sense of the least squares. 



4.1.3 Orthogonality Principle 

We saw that the minimum error-variance, linear, estimator, which is of the form 
(4.11), was unbiased, thus that EJX[ = 0. Let us prove now that, in addition, 
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Cov{x,y} = 0, 

i.e. that the measurement Y and the estimation error X = X — X are uncorrelated. 
The following holds readily: 

Cov{x,y} = e{(x-e{x})(y-^) t } = e{(x-x)(y-^) t } 

= E|[(X-M, x )-E A . y Z yy 1 (Y-M, y )][Y-(i y ] T | = E A . y -Z xy Z yy 1 5: yy =0. 

On the other hand, 

Cov{x,Y} = E{x(Y- l j, y ) T } = E{XY T }-E{x} l i];=E{XY T }. 

The comparison of these two equations implies that EJXY T [ = 0, thus that X 
and Y are also orthogonal. 

Reciprocally, let us prove that if a linear estimator X = KY + 1 is: 

1. unbiased, 

2. such that Cov{x,y} = 0, 

it is then an optimal linear estimator. 

The unbiased property implies, as proved in Sect. 4.1.2, that it is of the form 
(4.5), namely 

X= i i x +K(Y- l x 7 ). 
Furthermore, the fact that Cov I X, Y [ = yields that 

= Cov{x,Y} = E{(X-X)(Y- l j, y ) T } 

= E{[(X-^)-K(Y-j J L y )][Y-j Ay ] T } = X X¥ K Zyy . 



Thus: K = X xy Z yy . The optimal value of K is regained here. 

Remark 4.4. Here again it came to the same to assume X and Y orthogonal as to 
assume them uncorrelated. If indeed E I X Y T [ = , which is the condition of or- 
thogonality, the following is true: 
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CovJX, Y} = E{x(Y-M, r ) T } = e{xY t }-E{x}j4 = , 

since the estimator was assumed unbiased. X and Y are thus uncorrected. 

Conclusion: Principle of Orthogonality. The uncorrelation of X= XX and 
of Y , or their orthogonality, is thus a necessary and sufficient condition of the op- 

timality of X . 

Physical meaning of the principle of orthogonality. If Y enables estimating X, 
it is because there exists a correlation between X and Y. If this were not the case, 
the consequence would be indeed that ~L XY = , and the estimation would be re- 
duced to X= \l x . If now a correlation were remaining between (X — X) and Y, 
we could thus estimate the difference (X — X) , and therefore improve the estima- 
tion X . This would mean that the estimate was not optimal. Thus, if no correla- 
tion subsists between (X — X) and Y, it means that X is optimal, and vice-versa. 



4.2 Statistical Relations in State Space 



4.2.1 Introduction 

The state space representation of systems, considered up to now deterministic, will 
be extended in this section to the description of stochastic processes. 

More precisely, it will be shown that every Markovian stochastic process, as 
defined hereafter, can be considered as the solution of a stochastic difference or 
differential equation, according to its nature (discrete-time or continuous-time), 
and will thus be modelizable by the state of a linear system fed by white noise. 



4.2.2 Markovian Stochastic Processes 

A stochastic process X(m) is Markovian if its evolution after an instant t depends 
only on the value of \(t) at that time, and remains thus independent of its value at 
times t < t . 
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Definition 4.1. A continuous or discrete stochastic process X(t) is said Mark- 
ovian if, for an arbitrary sequence of times such as t k < ■ ■ ■ < t x < t < u , the condi- 
tional random variable 

x(«)|x(o,x(o,..,xfe) 

has the same probability distribution as the conditional random variable 

X(«) | X(t) . 
Remark 4.5. A Markovian process is also called a memoryless process. 



4.2.3 Discrete-time Markovian Process. Noisy Discrete 
Systems 

4.2.3.1 Stochastic State. Stochastic Linear Difference Equation 

Let us consider a discrete -time linear system, at the input of which a vectorial ran- 
dom signal V k is applied instead of a deterministic signal. 

The state of the discrete system becomes then a stochastic state X k and its dif- 
ference state equation becomes a stochastic linear difference state equation: 

X i+1 =0,X t +E t V t . (4.12) 

Remark 4. 6. This state equation is considered here in the general case of a linear 
system, not necessarily time-invariant. The matrix 

O k =0(t k+l ,t k ) 

is the transition matrix of this system from step k to step k + 1 , and therefore de- 
pends here on time (see Sect. A. 5, Appendix A). For the sake of generality, the 
same assumption will be made concerning E k . Such a system is represented in 

state space form in Fig. 4.1. 
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Fig. 4.1 Time-varying, linear, discrete-time system with random input and random state. 

4.2.3.2 Markovian Stochastic State 

Suppose now that the random process V k is white noise, satisfying therefore the 
following relations 

E{V t } = 0, 
Cov{V t ,V,} = Q t 5y. 

Of course, this noise is correlated neither with the present system state nor with 
any past system state: 



cov{v t ,x i } = o, yj<k. 



(4.13) 



It is easy to see that in this case the conditional random variable X k+l | X^ 
does not depend on the past values X ■ , Vj<k. 

The state X t , at given k, is then a Markovian stochastic state, and the discrete 
stochastic process X k defined by (4.12) is a Markovian stochastic process. 



4.2.3.3 Solving the Linear Stochastic Difference Equation 

The solution to (4.12) is thus as a Markovian stochastic process. Let us character- 
ize this process. 

The initial state X is itself random and is characterized by its mean \l x and 

its variance Z . The initial data of the problem are thus: 
E { x o} = ^x,o and Var{X }=Z . 
The process mean and its variance obey thus the following equations: 
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Mean value of the process: 

V-x.k+i=^{^k+i} = ^kV-x,k> (4.14) 

with given [i x . 

Consequence: the stochastic process X k can always be assumed centered. Indeed, 

- if [i xo = , we have from (4.14): \i xk = 0, M k ; 

- if \i, x0 * , it is possible to revert to the previous situation by a simple 
change of variable, by letting Z, k = X k — [x xk , the process Z k being then 
centered. 

Autocovariance of the process: Cov \Xj , X k \ = T, Jk . 

With the assumption \i x k — , the autocovariance is expressed as 

^=E{X y .Xj}. 
We calculate first Z /t+1 k : 

S i+1)i = E{x /fc+1 XJ} = E{(4> i X i +E i V yt )Xj} 

= «» t E{X yt Xj} + E /fc E{v i Xj}. 

The second term of the right side vanishes according to (4. 13), so that 

E i+u =^E i>t ■ (4.15) 

Applying repetitively this relation, from step £ + 1 until step y' ( j > k + 1 ), and 
using the fact that 

o ,o -, ... o, = o , . 

which is the basic property of the transition matrix (Sect. A. 5), we get: 

Z Jik =* Jik Z ktk , for j > k . (4.16) 

This equality writes also 

* Jtk =Y. Jtk T.£. (4.17) 
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It can be proved that this relation is characteristic of Markovian processes. 

Variance of the process: Var {X t } = Z t = E j X k XM . 

Let us establish how this covariance matrix "L k of the stochastic state X t is 
propagated from one step to the next, in absence of any measurement which might 
improve the knowledge of this state. According to (4.12), the following holds: 



-t+i 



E{x, +1 X t T +1 } - E{ (O k X k + E k V k )(O k X k + E t V,) T } 
: E {*, X k X J k «lt T + E* V t Xj */ + O k XXEj + E k V k V t T E J k } . 



Due to (4.13), this expression simplifies to 



Z t+1 =O t 5: t O/ + E t Q,Ej . (4.18) 



This equation is called the propagation equation of the variance or of the co- 
variance matrix. It has the initial condition: Z = Var{X } . 



4.2.3.4 State Model of a Discrete Markovian Process 

We can now inverse the problem, and consider that a given Markovian process is 
built by feeding white noise at the input of a linear system, to be determined. 
From (4.15) and (4.18) the following theorem is deduced: 

Theorem 4.2. Every discrete Markovian stochastic process X k , of mean \x x k 
and of covariance Z ■ k , can be represented by the state of a discrete linear sys- 
tem fed by a discrete white noise. 
This model is described by the linear stochastic equation 



with: 



X t+ i=^tX t +E t V t> 

E{X }=n X0 , Var{X } = I , 

E{VJ = 0, 

Cov{v„V ; } = Q t 8, ; , 

Cov{v„X y .} = Vj<k, 



and where 
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* = ^k+l,k ^k,k ■> 

E k Q k E T k =T k+l -® k T k tf 



k+\ v k '-'k ™ 

k+1 ~^k+l,k ^k,k ^k,k+\ • 



4.2.3.5 Generalized Discrete Model 

The previously considered system was a linear system fed by white noise. This 
noise V k affected the state X k of the system, which from deterministic had be- 
come random. For this reason, V k is called state noise or process noise. 

Let us complete this model by taking now into account the output y k of such a 
system, itself corrupted by a measurement noise W k . 

Consider the linear system, governed by the following state equations: 

Y t =C t X t + W t 

where O k , E t and \ k are defined as previously. 

The measurement Y k consists of a known linear function of the state, C k X k , 
perturbed by the discrete noise W k , which we assume again to be white noise. It 
has thus become also a random vector. 

The state X k and the measurement noise W k are uncorrected stochastic proc- 
esses, with the following means and covariances: 

E{X i } = M. J r.t. Cov{x i ,X y } = L t>y> (4.20) 

E{WJ = 0, Cov{w t ,W / } = R t 5 ty> (4.21) 

Cov{W t ,X y } = 0, yj,k. (4.22) 

The system described by (4.19) is represented in Fig. 4.2. 

Y k defined this way represents still a discrete stochastic, but no longer Mark- 
ovian, process with the two following moments: 

V-rjt = c k Vx,k > ( 4 - 23 ) 

Cov{Y i; Y y .} = C k L tJ C] + R k d kj . (4.24) 
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Fig. 4.2 Discrete stochastic system with process and measurement noise. 

The proof of (4.23) is trivial, the noise W t being centered. To prove (4.24), we 
write: 

Cov{Y k ,Y j } = E{(Y k - [ i r , k )(Y j - [ i 7J ) T } 

= E{[C,(X,-^ t ) + W t ][C i (X i -^ J ) + W y ] T } 

= c* e{(x 4 -\i Xik yXj -v-xj) T }c T j +C k E{(X t -M.x,t)Wj} 

+ E{w,(X i -kixj) T }c] +E{W,WJ}. 

Since W k is centered, the second and third term on the right side of this equation 

amount respectively to C k CovlX k ,Wj\ and Cov|W i .,X y -jC]' , and vanish thus 

because of (4.22). The proof is then completed simply by the use of (4.20) and of 
(4.21) for the two remaining terms. 

4.2.3.6 Case of Stationary Processes 

If we start from a stable discrete linear and time-invariant system, thus described 
by constant matrices O , E and C: 



X t+1 =*X t +EV, 

Y t =CX,+W t 



and if we suppose the white noises V t and W t stationary, i.e. 



Q /f =Q, R,=R, 



where Q and R are constant matrices, the equations of the previous sections are 
modified as follows: 
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Transition matrix: 

O jk =0(j-k) = 0(m), 

where m~ j — k . The transition matrix depends thus only on the time interval, 
expressed in discrete time steps. Recall that, for m = 1 , <S> k = O , and that, in the 
case of a discrete-time linear system resulting from the sampling at a constant pe- 
riod T s of a continuous-time linear system with system matrix A, this transition 

matrix amounts to = exp(A7' s ) , as in (A.23) in Appendix A. 

Consequences: 

1 . The stochastic process X k is stationary. 

Indeed, according to (4.16), T,j k is then, itself also, only function of j — k : 

2 Jtk = HJ-k), 

which implies that 

!(*,*) = I t>t = 1(0) = I, 

which is a constant matrix. 

2. Covariance matrix: 

(4.15) is therefore replaced by 

£ i+u =*Z, (4.25) 

whereas (4.16) becomes: 

r T1 [o M Z, if m>0 

E i+ m,t=E{x i+m Xn= ' " (4.26) 

1 ' [Z(0 T r ra if m<0 

Proof: 

if m > , by repetitive application of (4.25); 

if m < : by transposition of the equation corresponding to m>0, while let- 
ting p = —m > , we have 
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3. Variance: 

As to (4.18), it is replaced by 
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Z = OZO T + EQE T . 



(4.27) 



The rewriting of Theorem 4.2 for the present case is derived easily from these 
relations, and the same holds for the equations replacing (4.23) and (4.24). 



4.2.4 Continuous Markovian Processes. Noisy 
Continuous Systems 



4.2.4.1 Stochastic State Model. Stochastic Differential Equation 

Let us consider a continuous-time linear system, not necessarily time-invariant in 
the most general case, and fed by a continuous-time white noise. Its state equation 
becomes then a stochastic linear differential equation: 



X(0 = A(f)X(f) + E(f)V(0 , 



(4.28) 



where V(t) denotes a continuous-time white noise, uncorrelated with the system 
past state X(t) , thus satisfying: 

E{V(0}=0, 

Cov{V(t\V(t')} = Q(t)8(t-t'), 

Cov{V(0,X(f')} = Vt'<t. 

Such a system is represented in Fig. 4.3. 



V(0 



white noise 




E(0 



*Q 



x(0 



U 



x(0 




A(0 



Fig. 4.3 Time-varying, linear, continuous-time system with random input and random state. 
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4.2.4.2 Passage from Discrete to Continuous Case 

Rather than reestablishing the formulae corresponding to the case of a continuous 
system, we will derive them from the previous study by considering the continu- 
ous system as the limit of a discrete system whose sampling time T s would tend 
towards an infinitesimally small value: 

(k + 1)7; -kT s =T S = A* -f dt . 

Relations governing the passage to the limit. 

• State equations: 






-> X(t) + dX 

-> I + A(t)dt 

-> E(t)dt 

- v(0 



(4.29) 
(4.30) 
(4.31) 
(4.32) 



Noise covariance matrix: 



Q, 



Q(0 

dt 



(4.33) 



Proof. Let us write side by side the state equations for the two system classes: 

±(t) = ( ®^1 = A(f)X(f) + E(0V(0 

dt 



H+i 



-a> k x k + E k v k 



dX(t) = A(t)X(t)dt + E(t)V(t)dt 

X(t) + dX{t) = [I + A(t)dt]X(t) + E(t)dtV(t) 



Relations (4.29) to (4.32) result readily from the simple comparison of the two 
cases, X k corresponding to X(t) and X t+1 to X(t) + dX(t) . 

The case of the passage relation (4.33) is more delicate to handle. What we are 
looking for is the equivalence between the discrete sequence of white noise V k 
and the continuous process of white noise V(t), which is not physically realizable. 

Remark first a difference in dimensions between the two Q matrices. Indeed, 
the covariance matrices of the noise are, respectively: 

• in the discrete case: Q^ 6 kJ ; 

• in the continuous case: Q(t) d(t — t'). 



4.2 Statistical Relations in State Space 209 

Q k is therefore a covariance matrix. But, since the Dirac distribution S(t — t') has 
the dimension of the inverse of time, as shows its basic property I 6(f) dt = \ , 

*J —DO 

the matrix Q(t) is, on the contrary, a power spectral density matrix. Recall indeed 

that, in the case of a white noise, a power spectral density matrix can be converted 
to a covariance matrix by multiplying it with a Dirac distribution, as shown in the 
stationary case by (C. 19) of Appendix C, by letting there A^ (r) = H^ (r) (cen- 
tered noise) and O^- (w) = O^ (constant matrix, the noise being white): 

1 f°° 1 r 

^xx (t) = — ®xx M e ] T du> = Oja — e J T du> = <V 6(t) , 

2-7T J -oo 2"7W -~ 

where use has been made of the inverse Fourier transform of 1. 

The covariance matrix Q(t)5(t — t') has thus infinite elements. To obtain the 
continuous white noise process as a limit, for T s becoming infinitesimally small 
(dt), of a sequence of discrete white noise, it is necessary to increase the amplitude 
of the discrete impulses at the same time that their duration 7" s is decreased, while 
maintaining constant the power spectral density. Let us clarify this point. 

Since white noise is a centered process, its autocorrelation matrix is equal to its 
covariance matrix: 

• discrete case: Q k 5 kj = E l V^Vj [ ; 

• continuous case: Q(t) 5(t-t') = EJV(0V T (?')} = ^ vv (t,t'). 

To realize the equality of the power spectral densities in these two cases means, 
since in both cases we deal with white noises, thus having constant spectral densi- 
ties, to realize their equality for w = . According to (C.18), Appendix C, this 
amounts to equalizing the following integrals of their autocorrelation matrices: 



• discrete case: | Q t 5 kJ dt =1 Q k dt = Q k T s ; 

• continuous case: / Q(t) S(t — t')dt = Q(t') . 

The equality of these two expressions at the limit T s — > dt imposes the correspon- 
dence 

Q k dt ► Q(0. 

which is indeed (4.33). 
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4.2.4.3 Solving the Stochastic Linear Differential Equation 

Let us now solve (4.28). Since the continuous white noise V(t) is not correlated 
with the past state X(t) , it is easy to verify that here again the conditional random 
variable X(u)\X(t) , where t<u, does not depend on the past values X(^), 
V t x < t . 

The solution of (4.28) is therefore a Markovian continuous-time stochastic 
process X{t) . The initial state X is again assumed to have the mean \i x and 

the variance Z . 

Let us derive now from the discrete case, by means of the passage to the limit 
formulae (4.29) to (4.33), the two first moments of the process X(t) . 

Mean value of the process X(t) : 

From (4.14) we deduce by applying the first passage formula that 

V-x (0 + d \x, x (t) = [I + A(t)dt] \i x (t) , 
i.e. 

M0 = A(0M0- 

This equation has the same form as the one governing the free response of a 
linear system. Its solution is given by (Sect. A.5, Appendix A): 

MO = <I>M) (i^o . (4.34) 

Covariance of the process X(^) : 

We will calculate it from (4.16), where the discrete times t,,t k are simply re- 
placed by the times t x , t 2 belonging to a continuous time scale. Hence 

2(t 2 ,t l ) = 9(t 2 ,t l )2(t l ,t 1 ), fort 2 >ti, (4.35) 

which can also be written 

0(t 2 ,t l ) = -L(t 2 ,t l )-L~ 1 (t i ,t l ), for t,^, (4.36) 

if 2.(t,t) is invertible. 

Variance of the process X(t) : 

The variance propagation equation of the discrete case, (4.18), is transformed by 
applying the relations of passage to the limit of Sect. 4.2.4.2 and by omitting mo- 
mentarily the time dependence, in order to simplify the writing: 
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1 + dl = (I + A dt) 1 (I + Adtf + Edt — E T dt 

dt 

= 1 + Aldt + 1A 1 dt + A1A 1 {dt) 2 +EQE 1 dt . 
Neglecting the infinitesimally small term of second order, we obtain: 
dl = Aldt + lA T dt + EQE T dt , 



i.e. also: 



±(t) = A(0 l(t) + l(t) A T (0 + E(0 Q(0 E T (0 . (4.37) 



This equation is called linear evolution equation of the variance, or of the co- 
variance matrix. It is solved from the initial condition 1(0) = 1 . 
Particular case. If V(t) and X are Gaussian, the solution X(t) of (4.28) rep- 
resents a Gaussian-Markovian process. 

4.2.4.4 State Model of a Continuous Markovian Stochastic 
Process 

An analog approach to that of the discrete case leads to the following statement: 
Theorem 4.3. Every continuous-time Markovian stochastic process, X(t) , with 
mean [l x (t) an d with covariance l(t 2 , t { ) , can be represented as the state of a 
continuous linear system fed by a continuous white noise. 
The linear system is described by the equation 

X(t) = A(t)X(t) + E(t)V(t), 

where A(t) is defined by 

0(t 2 ,t 1 ) = A(t 2 )0(t 2 ,t,), 
with ®(t 2 ,t l ) = l{t 2 ,t l )l-\t l ,t l ), t { <t 2 , 

and where V(t) is defined by 

E{V(0} = 0, 

Cov{V(t),V(t , )} = Q(t)^(t-t , ), 

E(t)Q(t)ti T (t) = ±(t)-A(t)l(t)-l(t)A T (t). 
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4.2.4.5 Generalized Continuous Model 

The previous model is completed by the addition of a measurement equation: 



X(0 = A(f)X(0 + E(f)V(f) 
Y(t) = C(t)X(t) + W(t) 



with the following supplementary assumptions: 

e{x(o} = MO ; cov{x(t),x(t')} = -L(t,t') , 

E{W(0} = ; Cov{W(0, W(0'} = R(0 $(t - 1') , 
Cov{X(0,W(f')} = 0, Vt,t'. 

The system described by (4.38) is represented in Fig. 4.4. 



(4.38) 



v(0 

white noise 



w(0 

white noise \\ 




Fig. 4.4 Time-varying continuous-time stochastic system with process and measurement noise. 

The stochastic process Y(t) produced this way is no longer Markovian. It has the 
two following moments: 

M0 = C(0M0, (4.39) 

Cov{Y(0,Y(O} = C(0Z(M')CV) + R(0 %f-t') ■ (4.40) 

The proof of the second of these formulae is very similar to the one which has 
been given in the discrete case, equation (4.24). 



4.2.4.6 Case of Stationary Processes 

Again, the solution in case of a time -invariant system will be a stationary process: 
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X = AX+EV 
Y = CX + W 

where the white noises V(t) and W(f) are stationary, i.e. Q(t) = Q constant and 
R(7) = R constant. The previous theory is modified as follows: 



Transition matrix: 



0(t 2 ,t i ) = 0(t 2 -t 1 ) = 0(r) = e XT . 



Variance: 

where £ is a constant matrix. Equation (4.37) is replaced by 



AZ+ZA T +EQE T = . (4.41) 

Covariance: The equation (4.35) becomes now 

Z(r) = e AT Z if t > 

Z(T) = Ze- ATT ifr<0 . (4.42) 

Proof of the second formula. If r <0 , according to the first of these formulae, 
Z(— r) = e~ Ar Z , then, according to (C. 17), Appendix C: 



Z(r) = (e~ Ar Z) T = Z T (e" AT ) T = ZV A T = Z e" A T . 



since a covariance matrix is always symmetric ( Z T = Z ). 



4.3 Optimal Linear Filtering 



4.3.1 Discrete Kalman Filter 



As mentioned in the introduction to the present chapter, we will establish first the 
equations of the discrete Kalman filter, which is easier to present mainly for nota- 
tional reasons. 
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4.3.1.1 Presentation of the problem 

We want to estimate the stochastic state vector X , of dimension n, of a discrete 
linear system submitted to disturbances (process noise V ), by means of the meas- 
urement vector Y , of dimension q, in the presence of a measurement noise W in- 
dependent from the state X. 

The state space representation of a linear system affected by such noise sources 
is thus the following: 

X, +1 -O t X t +E t V, 
Y t =C t X t +W t 

where it is assumed that the process noise V k and the measurement noise W k are 

distinct white noise sources, thus not correlated with each other. The assumptions 
concerning these equations can thus be summarized as follows: 

E{V t } = 0, Cov{v k ,Vj} = Q k 5 kj , 
Cov{V t ,X y } = 0, Vj<k, 

E { X o} = M-x,o> Var{X } = Z , 

Var{Xj = Z i ., 

E{W,} = 0, Cov{w t ,W,} = R t 8 y> 

Cov{W A ,X i } = 0, \/j,k. (4.44) 

By definition, Q A . is a symmetric, positive semidefinite matrix (Appendix C, 
Sect. C.2.6). We assume in addition that R k , also symmetric, is positive definite, 
i.e. that the various measurement noises are statistically independent, which is a 
sound hypothesis if the measurement sensors are distinct. We assume further that 
the pair (O k ,C k ) is detectable Mk , i.e. that the model contains no unstable and 

unobservable eigenvalue. 

This system will thus be represented by the generalized discrete model, illus- 
trated in Fig. 4.2. 

The present objective is to determine the best estimate of the state X k , in the 
sense of the minimal error variance, by using only the measurement sequence 

v o> v i> ■•• ' y/t-i>y* • 

Define: 
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• x k \ k as the optimal estimate of X k elaborated after the measurement y^thus 
by taking into account all the measurements of the above sequence; we will call 
it in the sequel estimation of the state X k ; 

• x t | t _j as the optimal estimate of the same state X k elaborated before the 
measurement y k , i.e. from the measurement sequence y ,y l5 ... ,y t _[ ; we will 
call this estimate prediction of the state X k . 



4.3.1.2 Estimation and Prediction Equations 

In order to establish the discrete Kalman filter equations, we will proceed initially 
in three phases, which will merge afterwards into only two phases. 

1 st phase. Consider the system at step k — 1. 

X t _j and V t _! being random vectors, the state vector X t of the next step will be 
random also. The best estimation of it that it is possible to make at this stage of 
knowledge, x^., , is its mean value or expectation 

%\k-\ = E { x t} = M'x,t • 

The estimation error being given by X^., = X t — \ k \ k _ { = X k —\i xk , it is evi- 
dent that this estimation is unbiased E jx t | t _j } = [i x k — [i x ,k = • ^ ts variance 
amounts thus to 

Var {X t|t _, } = E {x t|t _, xj t _, } = E { [x k - yi x> k ][x k - y, x k f } = Var {X t } . 

X jt | t _ 1 being the estimation error of X t before the measurement y k , i.e. from the 

measurement sequence {yo>yi>--->y,t-i} ! its variance Varjx t | t _j[ will be de- 
noted by E t | t _i , using the same notation as the one adopted for the estimates. 
According to (4.14) and (4.18), the previous values are given respectively by 

V-x.k-^k-iV-x.k-i' 

which can also be written 

**l*-l=*U**-ll*-2> ( 4 - 45 ) 

and by 
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St,*-, = <*>,_! S t _,| t _ 2 #_, + E t _! Q t _, Ej_j . (4.46) 

2 nd phase. Suppose now that the transition at step k takes place, and that a meas- 
urement occurs after the transition. This measurement yields a realization y k of 
Yj. = Cj. Xj. + Wj. , with the assumptions given at the beginning of this section. 
According to (4.4), the best estimate of X k is then given by 



*k\k=V-X,k+ K k<Jk-V-T,k) 

= **l*-i + K *(y* -CA\k-i), (4-47) 

since, at the present time, our best estimate of X k is still \i xk =x k \ k _ 1 , from 
which results, by using (4.23), that \i Yk = C k \i, xk = C k x k \ k _ x . In order to calcu- 
late K k , given by K^ = E^y X^ according to (4.8), we need to evaluate the ma- 
trices Z ry and ~L XY . A calculation similar to the proof of (4.24), applied here for 
j = k , yields successively 



Lyy 



■■ Cov {Y k ,Yj } = E |[c t (X t - x klk _, ) + W k \[c k (X t - x klk _ { ) + W t f } 
= C k E{(X k -x klk _ l )(X k -x klk _ l ) J }c J k +E{w k W k 1 } 

where use has been made of (4.44). Similarly: 

X XY =Cov{X k ,Y k } = E{(X k -x klk _ l )(C k X k -C k x klk _ l +W k ) T \ 

1 ' (4.48) 

— ^klk-l^k 



Finally, 



K k = Z t|t _, Cj(C, I t|H Cj + R,)" 1 • (4.49) 



Now that Kj. is calculated and x t | t determined by (4.47). It is clear that the 
variance of the estimation error X k \ k = X k — x k \ k , which corresponds to this new 

estimate, will also have a new value, which will result from the application of 
(4.10): 
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2*1* = Var{x i|t } = 1^ - E t|t _, Cj(C, Z t|t _, C^ + R k y l C k L t|t _, ,(4.50) 

where the last term, I H = C t S t | A: _ 1 , has been obtained by transposition of (4.48) 
and application of the symmetry property of any covariance matrix. 
Remark 4.7. Recall that S tit— i represents the estimation error variance Z>e/ore the 
measurement, and ~L k \ k the same variance after the measurement. Equation (4.50) 
shows then that the latter is never superior to the error variance before measure- 
ment. The term T k \ k _ l Cj(C k 'L k ^ k _ l C k : + 'R k )~ l C k 'L k \ k _ l expresses indeed the 
diminution of this variance due to the improvement that the measurement y k 
brings to the knowledge of the state \ k . 

Remark 4.8. Equivalent expression of K k : 

K^I^C^R- 1 . (4.51) 

Proof. Let us right multiply both sides of (4.50) by C^ R^" 1 , and then factor in 
the right side the term Z t | t _] C k (C k T, k \ k _ 1 C k +'R k )~ 1 to the left and the term 
R t _1 to the right. This yields successively 

^t|t*-"t-**t 

= ^t|t-i*-'t-**t — ^t|t-i *--t(C.t ^t|t-i ^t +"t) C k Zi k ^ k _ l C k R k 

= '^k\k-i C k( C k^'k\k-i C k + R t) \( C k^'k\k-i C k +Rk)- C k'Zk\k-i C k\ R k 
= ^k\k-i C k( C k^k\k-i C k+^-k) [^■kl^-k 

The last expression is equal to K k according to (4.49). 

3 rd phase: recursive algorithm and improvement of phase 1. In the hypothesis 
of a recursive algorithm, a possible improvement of the previous equations seems 
evident. Nothing indeed kept us at step k — 1 , thus in the first phase of this algo- 
rithm, from using the estimation Xt-ilt-i °f me state of step k — \, instead of its 
prediction \ k _ 1 =x jt _ 1 | jt _ 2 , to predict the state of step k, since the measurement 
y^.j had then already occurred. Equation (4.45) is then replaced by: 



l £l£-l 



: ®t_iXt-ilt-i- (4-52) 
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A similar reasoning leads to the replacement of (4.46), which describes the propa- 
gation of the estimation error variance from step k — 1 to step k, by the following: 

-L k]k _, = <&,_, Z t _ 1|t _, C + E t _, Q t _, Ej_, , (4.53) 

since here again we can take benefit from the improvement which occurred in this 
variance after determination of the new estimate of step k — 1 . 



4.3.1.3 Recapitulation: Discrete Kalman Filter 

The Kalman filter is thus represented by a set of recurrent equations, which can be 
grouped in an estimation phase, corresponding to equations (4.47), (4.49) and 
(4.50), and a prediction phase, which corresponds to equations (4.52) and (4.53), 
these two last equations being rewritten here for the step k + 1 : 



Estimation (or update) phase: 

»*l* =»*l*-i +K t (y*-C i x 4 | t _ I ) (4-54) 

K k = E^ Cj(C t Vi C T k + R,)" 1 = Z t|t Cj R; 1 (4.55) 

Var{x t|t } = Z t|i = L i|t _, -Z t|t _, Cj(C t I i|t _, Cj + R,)" >C t Z t|t _, (4.56) 
= (I-K i C i )E i|t _ 1 (4.57) 

Prediction (or propagation) phase: 

x t+ i|i=^fcXiit, x |_! given (4.58) 

Vii* = °* E *l* * T + E * Qt E I • Vi S iven (4-59) 

4.3.1.4 Definition of Innovation 

The estimation at step k of X k according to (4.54) takes into account all past 
measurements and the present measurement y k . This last measurement brings re- 
dundant information; in reality, the new information about the state X t is con- 
tained in 



4.3 Optimal Linear Filtering 219 

v k = yt ~Ck x k\k-i > 

quantity called innovation. In other words, the innovation represents the difference 
between the present measurement y k and the predicted measurement C jt x t | jt _ 1 . 
Note of course that v k is a realization of the following random variable N k : 

N t =Y t -C t Vi. (4-60) 

4.3.1.5 Physical Interpretation of the Kalman Filter 

The Kalman filter (equations (4.54) and (4.58)) is thus a model of the system, as 
described by (4.43), with a correction term proportional to the innovation. 

It is interesting to establish a parallel with the physical interpretation which has 
already been given about the Luenberger observer, in Chap. 2. In fact, the Kalman 
filter is nothing else than a Luenberger observer considered in a stochastic context, 
thus with addition of noises. 

Furthermore, one notes that the proportionality matrix K t given by (4.55), 

also called gain matrix of the filter, is essentially the ratio between 

• the uncertainty in the state, Z t | t , 

• and the uncertainty in the measurement, R t , 

the matrix C k expressing simply the sensors which build the measurement Y k 
from the state X t . 

This permits understanding better the operation of the Kalman filter, underlying 
to (4.54): 

• if the measurement noise is important, the gain K t will become very small, 
and priority will be given to the model simulation; 
conversely, if the state equation disturbances are important, a gain K t which 

becomes then high will strengthen the influence of the measurements on the es- 
timations. 



4.3.1.6 Condensed Form of the Estimation and Prediction 
Equations 

There are two ways of condensing the two previous phases, of estimation and pre- 
diction, in one unique phase. 



• 
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Estimator filter. This form is obtained by elimination of x^^ between (4.54) 
and (4.58), which yields: 



k il(t 



: <^-ii i -il*-i+K i (y i -C 4 4| t _ 1 i 4 _ 1 | i _ 1 ) . (4.61) 



Joined to the equations (4.55), (4.57) and (4.59), this form leads to the symbolic 
block diagram of the discrete Kalman filter, illustrated in Fig. 4.5. 

Predictor filter. This is the most interesting condensed form for practical appli- 
cations. By eliminating this time \ k \ k between (4.54) and (4.58), we obtain: 

f *+il* =®k%\k-\ +«9t K *(y* -C t i t | t -i) • (4.62) 

Likewise, the elimination Z t , t between (4.56) and (4.59) yields: 



'Z k+l]k —® k 'L k]k _ 1 <b k -® k 'L k]k _ l C k (C k 'L k]k _ l C k + R t ) CfcEj.^!^ 

+E k Q k E T k .(4.63) 

This equation is again a Riccati (matrix) difference equation, very similar to the 
one already encountered in Chap. 3. The duality between these two equations will 
be discussed again in Sect. 4.6. This duality allows showing that its solution 
X /t+1 i t is a positive semidefinite matrix. A criterion of positive semidefiniteness is 

given in Appendix B. 



4.3.1.7 Filter Implementation 

Initialization. Consider that the measurements are received from the time t = . 
Suppose further given the expectation and variance of the random initial state X : 

E{X } = (X X0 ; Var{X } = E{(X -M, xo )(X -M, xo ) T }=E . 

Two cases can then occur: 

1. If the first available measurement is y , choose x |_, = [i xo and S |_j = £ ; 

the first step will then be that of estimation at k = by using successively 
(4.55), (4.54) and (4.57); this is the situation depicted in Fig. 4.5; 
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Fig. 4.5 Functional and algorithmic diagram of a discrete Kalman filter, 
associated with a discrete plant with disturbances. 



2. If the first available measurement is y l , choose x | = [x xo and Z | = X ; 

the first equations used will then be those of prediction of step k — l: equations 
(4.58) and (4.59) applied for k = . 

Preliminary computations. Neither Z t , t nor K t depend on the measurement 
y k . They can therefore be calculated in advance and stored in the computer mem- 
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ory, in the form of sequences {2 t | t | and {K t }. The computation of x k \ k from 
y, is then very simple, which facilitates the real time implementation of the filter. 

4.3.1.8 Properties of the Innovation Sequence 

The sequence of innovations {N^Nj,.--,^} , generated by the Kalman filter, 
constitutes a discrete white noise, of variance 

Var{Nj = C i E i|i _ 1 Cj+R /fc . (4.64) 

Proof. The first result stems from the fact that N k depends only on Y k , and not 
on the past measurements {Y , Y 1 ,..., Y t _j} . On the other hand, N t is also in- 
dependent of the quantities -j x^q,^!! ,. - .,x A | /t _ 1 \ , which are functions of the above 
measurements through (4.62). N k is therefore independent of the past innovations 
N ■ = Yj — CyX 7 |y_j , V j < k . In conclusion, N k is a white sequence: 

Cov{N t ,N,} = 0, Vj*k. 

This result is intuitive, since if N i+1 were correlated with N k , this would 

mean that not all the information available to elaborate x k \ k would have been ex- 
tracted from y k , and that this estimate was therefore not optimal. A similar rea- 
soning has already been made to give a physical interpretation of the principle of 
orthogonality, in Sect. 4.1.3. 

It is thus possible in theory to check the optimality of a filter by applying an 
experimental test of whiteness to the observed innovation. 

Furthermore, N k can also be rewritten as N k = C k (X k —x k \ k _ l ) + W k , which 
proves, with the help of (4.19) and of (4.24), the second half of the stated property. 



4.3.2 Continuous Kalman Filter 



4.3.2.1 Position of the Problem 

Consider a continuous-time linear system, time-varying in the most general case, 
and submitted to process noise as well as to measurement noise. 



4.3 Optimal Linear Filtering 

It is represented by the following equations: 



X(0 = A(0X(0 + E(0V(0 

Y(0 = c(0X(0 + w(0 
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(4.65) 



where again V(t) and W(7) are supposed to be white noises, here continuous- 
time, not correlated with each other. In addition, V(t) is supposed not correlated 
with past values of X(t) and W(7) not correlated with X(t) at any time. 
The problem hypotheses are thus: 



E{X } = M , X;0 , Var{X } = Z 
E{V(0} = E{W(0} = 



v« 



Q(0 _o 

R(0J 



8(t-t') 



(4.66) 



v(0 
Nw(0j(w(O 

e{v(ox t (?')} = o, \/t'<t 

e{\V(0X t (?')} = 0, W/ 



As in the discrete case (Sect. 4.3.1), Q(t) is a symmetric, positive semidefrnite 
matrix (see Sect. C.2.5), and we assume furthermore that R(?) , also symmetric, is 
positive definite. Again we assume that the pair (A(t),C(t)) is detectable W , i.e. 
that the model does not contain any unstable and unobservable eigenvalue. 

Let us determine the best estimate x(t) of the state X(t) , in the sense of the 
minimal estimation error variance, from the measurements y(r), r G [0, t] . 



4.3.2.2 Derivation of the Equations by Passage to the Limit 

Kalman gain matrix. Let us recall here (4.55) of the discrete case: 



K, 



-•k\k-\ 



C k( C k^k\k-l C k + R t) 



The variances of the estimation error before and after the measurement ~L k \ k _i and 



■•k\k 



of the discrete case will be replaced here by Z(? ) and Z(? + ) , where t 



and t + will denote now instants, respectively just before and just after the meas- 
urement y(t) occurs. 
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It is clear that, when the time interval dt separating these two instants will tend 
to zero, t~ and t + will merge with each other, and that 



i(r)^i(t + )^i(t) 



will hold, i.e. also: 



-•k\k-\ 



■•k\k 



■l(t). 



By passage to the limit, the following will hold for K k , by using (4.33) for R k 



K, 



>mc\t) 



C(t)i(t)c\t)+^ 
dt 



>l(t)C T (t)[C(t) l(t) C T (t)dt + R(0 1 dt 

>l(t)C J (t)R-\t)dt 



The last line of this passage to the limit was obtained by neglecting the infinitesi- 
mally small term in dt in the brackets before the finite term R(?) . The following 

equivalence relation, between discrete and continuous case, is thus obtained for 
the Kalman gain: 



K k >K(t)dt , 



(4.67) 



with: 



K(t) = l(t)C T (t)R-\t) 



(4.68) 



State estimation. There is therefore no more reason to distinguish here between a 
prediction and an estimation phase, these two phases having merged. At the pas- 
sage to the limit, the correspondences will become 



v k\k-l 



*-k+l\k 



k /d/ l - 



*-k+l\k+l 



x(0 
■ x(t) + dx(t) . 



Starting e.g. from the condensed form (4.62) of the predictor filter of the dis- 
crete case, 



u+iit 



: ®***i£-i +«% K *(y* -cAit-i) . 
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the following is obtained, by applying the passage relations (4.29) and (4.30): 

x(0 + d x(0 = [I + A(0 dt]x(t) + [I + A(t) dt]K(t) dt [y(0 - C(t)x(t)] . 

By subtracting x(t) to the two sides and neglecting in the right side the terms 
which are infinitesimally small in dt 1 , we get finally: 



dx(t) 
dt 



x(0 = A(t)x(t) + K(0[y(0 - C(0x(0J . (4.69) 



Continuous propagation of the covariance. Let us start again from the con- 
densed form (4.63) of the discrete case: 



J t+l|i ' 



:« !t Z t|t _ 1 «i t T -^nt-iCl(C t Z t|t _ 1 Cj+R t )- I C t Z t|t _ 1 ^ l T +E t Q t Ej 



As above, since the two phases have merged, the following correspondences can 
be established at the passage to the limit: 

^k\k-i ~^ ^k\k ~* m*) 
z* + ii*-z t+ n*+i -+?#)+ dut). 

By using the passage relations (4.30), (4.31), (4.33) and (4.67), by omitting 
temporarily the time dependencies to simplify the writings, and by neglecting 
again the terms which are infinitesimally small in dt of order greater than one, the 
following equations result successively: 

?. + d?. = (\ + Adt)(\-KCdt)?.(\ + Adtf + Kdt^- E T dt 

dt 

= (I + Adt-KCdt)I l (I + Adtj r + Vdt—ti T dt 

dt 

= I. + ALdt+ IA r dt - KCE dt + EQE 1 dt 

Hence finally, after reintegration of the time dependencies: 

t(t) = A(t)X(t) + X(t)A J (t) - E(0C T (0R "' (t)C(t)X(t) + E(0Q(0E T (0 .(4.70) 

This differential equation, which is nonlinear in Z , is a matrix differential Ric- 
cati equation, also already encountered in Chap. 3. It can be shown here also that 
its solution ~L(t) is a positive semidefinite matrix. 
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Some physical interpretation can be given for this differential equation. The 
time variation of the error variance, i.e. of the estimation uncertainty, is due to the 
contribution of three terms: 

1 . A(?)E(7) + Z(?)A T (t) results from the behavior of the homogeneous system 
(free response) without measurement; 

2. Q(t) expresses the uncertainty increase due to the process noise (state noise); 

3. —l l (t)C J (t)R~ l (t)C(t)'L(t) expresses the uncertainty diminution resulting 

from the measurements. In the absence of measurement, indeed, the following 
equation is obtained: 

Z(0 = A(0S(0 + l(t)A T (f) + E(0Q(0E T (0 , 

which is the evolution equation of the variance (4.37), describing its free evolu- 
tion, without improvement term. 



4.3.2.3 Innovation 

Similarly to the discrete case, it can be shown that the innovation 

N(0 = Y(0-C(0x(0 

is a white noise, thus centered, and that its variance equals the one of the meas- 
urement noise 

cov{n(o,n(0} = R(0 Sdt-t 1 ) . 

It can thus be used here also to test the optimality of the filter (see Sect. 4.3.1.8). 

4.3.2.4 Recapitulation 

The continuous-time optimal filter, also called Kalman-Bucy filter, is thus given 
by the following equations: 

Equations: 

£(0 = A(0£(0 + K(0[y(0-C(0£(0] (4-71) 

K(t) = -L(t)C J (t)R-\t) ( 4 - 72 ) 

i(t) = A(0S(0 + E(0A T (0 - Z(0C T (0R "' (t)C(tWt) + E(?)Q(0E T (0 ( 4 - 73 ) 
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Initial conditions: 



x(0) = ^ 



Z(0) = E 



Remark 4.9: Initialization. If the initial state is perfectly known, the choice should 
be of course [i x = x and E = . If this is not the case, a plausible initial state 

should be chosen and a very large matrix Z should be associated with it. 

The representation of such a filter in Fig. 4.6 shows that it realizes a simulation 
of the plant (without noise), calculates the estimated quantities x and y , com- 
pares y to the true measured value y, and feeds back the difference into the filter 
through the gain matrix, K. 



White noise ~W(t) 




Fig. 4.6 Continuous-time system and associated continuous-time Kalman filter. 



4.3.3 Asymptotic Behavior of the Kalman Filter 



We will consider now the stationary case, where 

• the plant is (stable and) time-invariant: A (or O) , E, and C are constant; 

• the noises are stationary processes: Q and R are constant. 

In such a case, the filter can reach, after extinction of the transient phase, a 
steady state, in the sense that "Lit) becomes a constant matrix, t,(t) = , for the 
continuous filter, while l, k+l ^ k = 'L k ^ k _ l ='L~ and l. k+ ^ k+l = Z t | t — E + for the 
discrete filter. 
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In the continuous case, the differential Riccati equation (4.70) is then replaced 
by the following algebraic Riccati equation: 

AZ + ZA T -ZC T R- 1 CZ + EQE T = , (4.74) 

in which all the time dependencies have disappeared. 

In the discrete case, the difference equation (4.63) becomes the following alge- 
braic equation: 

I"=OrO T OZ-C t (CZ _ C t + R) 1 CZ-O t +EQE t . (4.75) 

Remark 4.10: Convergence conditions of the solution of these two equations. By 
duality with the optimal control, the two previous equations have, each respec- 
tively, one unique positive semidefinite solution if [Duc02] : 

1. the pair (C,A) , respectively (C,0) , is detectable; 

2. the pair (A,Q ) , respectively (O,Q ) , is stabilizable, where Q is an arbitrary 
matrix satisfying EQE T = Q Qj . 

If in addition (A, Q ), respectively (O,Q ), is controllable, X , respectively X~, 
is positive definite. 

Remark 4.11. The solution of the two equations permits in theory to calculate Z 
or 5T in steady state. In practice it is however often preferable to integrate nu- 
merically (4.70), in the continuous case, or to solve iteratively (4.63) in the dis- 
crete case, with Z = , until the steady state solution is reached. Another method 
consists in using, by duality, the Hamiltonian matrix method discussed in 
Sect. 3.5.6. 

According to (4.72) in the continuous case or (4.55) in the discrete case, the 
following equations hold in steady state: 

K(?) 1 JZCR" 1 (continuous) 

or K J k ~ |i;-c T (CZ-C T + R)" 1 = £+C T R ! (discrete) (4J6) 



where K is a constant matrix. In the discrete case, the estimation error variance af- 
ter measurement becomes then: 

Z + =(I-KC)5T . (4.77) 

The optimal filter is then time-invariant and is governed, in the continuous case, 
by the following equation: 
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i(0 = (A-KC)x(0 + Ky(?) . (4.78) 

This equation reflects, in the time domain, the Wiener filter, generated in the 
frequency domain by solving the Wiener-Hopf integral equation. It is possible to 
say therefore that the Kalman filter represents an extension of the Wiener filter to 
time-varying systems and to non stationary noise sources. 

Remark 4.12. The constant value K of the Kalman gain corresponding to the 
steady state is sometimes used in place of the exact solution K k or K(t) , if it is 

desired to simplify the on-line computations. It should be noted however that such 
a stationary filter will have a different behavior than the Kalman filter described 
previously, during the transient phase resulting from the initialization or from any 
sudden change occurring later in the plant state or parameters. During such tran- 
sients, the stationary filter will not weight correctly the information brought by the 
initial estimate and the first measurements. 



4.3.4 Generalization. Addition of a Deterministic Term 

The linear systems considered up to now, continuous or discrete, were fed by only 
one input signal, which was supposed random. 

Equally, the direct feedthrough matrix D, which adds to the output equation of 
a state space represented system a term directly proportional to the input, was as- 
sumed to vanish. 

The following equations show briefly the modifications which take place in the 
previous equations, when a deterministic input vector u k is added to a discrete 
system, 



k+\ ~ 


= <t> k x k +r k u k +iL k v k 


Y t = 


= C k X k +T> k u k + W k 



(4.79) 



and when a deterministic input u(t) is added to a continuous system: 

fX(0 = A(0X(0 + B(0u(0 + E(0V(0 
Y(0 = C(0 X(0 + D(0 u(0 + W(0 

It is easy to see that the only changes in the equations of the Kalman filter consist 
in the introduction of a term containing u in (4.54) and (4.58) in the discrete case, 
and in (4.71) in the continuous case. Fig. 4.7 illustrates this situation. 
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White noise 

w t ,w(0 




Fig. 4.7 Continuous-time and discrete-time plant with process and measurement white noises 

and deterministic input. 

Let us consider successively the two cases. 
Discrete case. The estimation equation (4.54) must be completed as follows: 



X k\k — X k\k-l + K t( V /t -Ci x /t|i-l ~ D A U i) • 

The corresponding equation yielding the best estimate of the output is: 



(4.81) 



y t|t =C t x t | t +D t u t . (4.82) 

The prediction equation (4.58) becomes 

*k+i\k =< bk%\k+T k n k , x |_! given, (4.83) 

and the condensed equation of the estimator filter, (4.61), is replaced by 



X k\k — %-l X k-l\k-l 



■ r t _!U t _, + K k y* - C* (%_! x*_i|*-i + r^u^! ) - D t U J , 



(4.84) 



whereas that of the predictor filter, (4.62), becomes 



^- + Ht=« > tX A | A _ 1 +r /t u t +0 A K A (y /t -C A x i .| i ._ 1 -D A u / ,-) . (4.85) 



Continuous case. Equation (4.71) becomes here 
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x(0 = A(0x(0 + B(0u(0 + K(0[y(0-C(0x(0-D(0u(0] • (4.86) 

Steady state in the stationary case. Equation (4.78) is replaced by 

x(0 = (A-KC)x(0 + Bu(0 + Ky(0-KDu(0 . (4.87) 



4.3.5 State Representation of the Discrete Predictor 
Kalman Filter 

The substitution of (4.81) into (4.82) yields: 

y^=C k x k]k _ 1 +C k K k (y k -C k x k]k _ 1 -D k n k ) + B k u k . 



k\k 



Rearranging somewhat the terms of this equation as well as those of (4.85) leads 
to the following set of two equations: 



W = ^t(i-K t c t )i t | t _ 1 + (r t -o t K,D t K +¥Ji 

( y klk =C k (I-K k C k )x klk _ 1+ (D k -C k K k D k ) ak +C k K k y k 



(a) 
(b) 



It is easy to recognize that these two equations are respectively the state equa- 
tion and the output equation of a linear system, whose state would be the predic- 
tion x t | t _j of the state of the observed system and whose output would be the es- 
timate y . of the output of this system, after update (estimate after 

measurement). 

The equation yielding the updated estimate of the state of the observed system 
(equation (4.81) above) can be added to this representation, also after rearrange- 
ment: 

x^KI-K.CJx^-K.D^ + K.y, . (4.89) 



4.3.6 Case of Colored Noise 

In all the preceding sections we have assumed that the noise sources applied to the 
plant, V k or V(t) , were white noises. 
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What to do in the case the input noise is a colored noise, therefore a correlated 
noise? This name will be given to a noise whose power density spectrum is no 
longer flat, thus whose autocorrelation function does not vanish for r ^ or 

The solution consists then in reverting to the previous problem, by representing 
the colored noise process by an appropriate linear system, whose input is fed with 
white noise (see theorem 4.2 or 4.3). The initial state vector is then augmented, by 
adding to it the components of this supplementary linear system (or filter). The re- 
sult is an augmented system, with white noise at its input. The Kalman filter is 
then calculated for this augmented system. 

In fact, if the spectrum of the colored noise is constant over a broad enough 
spectral bandwidth, as compared with the bandwidth of the considered system, or, 
equivalently, if its autocorrelation function is narrow enough, as compared to the 
dominant time constants or response time of this system, this colored noise can be 
represented to a reasonable approximation by white noise. 



4.4 Covariance Matrices of the Noises 

4.4.1 General Considerations 

In the case of white noises, they are characterized by their covariance matrices: 
Q = Cov{V t ,V t } = E{v,V t T }, and R = Cov{W„W t } = E{w t W?} . 

There is usually no information as to eventual correlations between the compo- 
nents of disturbances acting on the various state variables. Therefore diagonal ma- 
trices will be selected for Q and R , the diagonal elements of which represent the 
variances of the components of these disturbances. 

4.4.2 Choice of the Process Noise Covariance Matrix 

In the majority of situations, the discrete-time plant models stem from the sam- 
pling of continuous-time plants at some period T s . The process noise acts on the 

continuous-time part of the plant. By assuming it to be white noise, it varies con- 
siderably during one sampling period. Therefore, its effect at the end of such a pe- 
riod cannot be determined the same way as we had done for equation (4.59): it is 
necessary on the contrary to integrate it. 



4.4 Covariance Matrices of the Noises 233 

By supposing, in order to simplify the writings and without any loss of general- 
ity, that the plant is time -invariant, thus that O k — O , T k = T , E(t) = E , equa- 
tion (A.22) of Appendix A, which served to establish the state equation of a sam- 
pled data system, namely 

x t+1 =e A7 <x t + e [( >s 'Bu(r)rfr, 

«-/ kT s 

permits calculating the conjugated effect of deterministic and stochastic inputs af- 
ter one period. Starting from (4.80), the evolution of the state over one period is 
given by 

T 

X t+1 =OX t +ru t + / " 5 e A(r '- r) EV(T)Jr, 

equation which replaces (4.79). With the use of (4.83), the error of the estimation 
of x t+1 | t before the measurement y k+l , thus in the prediction phase, can be writ- 
ten: 



X k+i\k = X k+i~*k+i\k =* VA k-t- > "k-t- I c nvyijui -*i t | t -' "k 



OX k +r» k + f S e A(T >- T) EV(T)dT-®x klk -T» k 
Jo 

T 

= 0(X k -x t]k ) + f ! e Affi - T) EV(r)rfr 
Jo 



The variance of this estimation error is given by S jt+1 | t = ^\X k+1 \ k X k+l \ k \, the 
estimate being unbiased, i.e. 



^•k+l\k — E 



U(X k -x kik ) + J o S e A ^- T) EV(T)dr\ 

T 1 T 1 

®(X k -x k]k ) + J q S e A ^- T) EV(T)dr\ 

O t + f ' f ' e A(r '" T) E E{v(r)V T (r')}E T e AT(r '- r,) dr'dr . (4.90) 

The noise V(t) being white noise, assumed stationary, it satisfies 

e{v(t)V t (t')} = Q8(t-t'), 

where Q is its power spectral density (see Sect. C.2.6). This yields 
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£ E{V(T)VV)}EV T ^'> dr> = £ Q8(r - T ')E V^ 3 ^ dr> 



= qeV {t *- t \ 

and (4.90) becomes: 

X, +1|t =OZ i1t O T +r r ' e A «^EQEV T «-^r. 
Jo 

Assuming further that, as is usually the case, the sampling period T s is signifi- 
cantly shorter than the shortest time constant of the plant, which implies 

the previous equation becomes: 

z t+1| ^os: t|i o T +EQE T 7;, 

or, after reintegration of the eventual time dependencies of the various parameters: 

z t+ n* ^ fl^tf +e(0Q(0e t (0t; ! , 

f^ s A 

which, with the equivalence E, = / e r E(r)<iT ~E(?)7' for small T s , can be 

Jo 

written 

Z A+1| ^O t Z i|t O A T +E t ^E t . (4.91) 

By comparison of this equation with (4.59) the passage relation (4.33) has been 
regained here: 

It is then possible to apply the prediction equation of the variance (4.59) of the 
discrete case to the case where V(t) is continuous-time noise, by using the ap- 
proximation Q A . ~ Q(t)/T s , under the hypothesis that T s is much smaller than the 

plant time constants. 

The white noise does not exist in reality, where rather "filtered" white noise is 
encountered. It has been shown that for the system below the following properties 
are true: 
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White noise, with 
power spectral ■ 
density Q 



i ) fc F(Q 



1 + ^J 
1. Autocorrelation function of the output noise, or real physical noise V(t): 



t \ 2 -\t\It c 



where t c represents the autocorrelation constant of the physical noise and 
where 



a 1 = yyy (0) = E {v 2 (t)} = Var {V(t)} , 



since V(t) is centered. 

2. Relation between the power spectral density of the white noise and the quad- 
ratic mean of the physical signal: 



Q = 2T c a 2 =2T c E{v\t)} 



The discrete noise covariance matrix, which enters equation (4.59), will thus 
have, according to (4.91), the diagonal element ii 

2 



^)n=y\ T A V2 ^h 



which involves the measured mean quadratic value, and the measured or esti- 
mated autocorrelation time c< 
component of the state vector. 



mated autocorrelation time constant of the disturbance which acts on the ;' 



4.4.3 Choice of the Measurement Noise Covariance 
Matrix 

Here, on the contrary, the noise is most often discrete -time noise, stemming di- 
rectly from the inaccuracy of the sampled sensors. It is white noise, therefore un- 
correlated from sample to sample. The simplest value to take for the diagonal ele- 
ments of the matrix R is the square of the mean quadratic error of the 
corresponding sensor, or square of its RMS value, 

R u =[E{w 2 (t)}l. (4.92) 
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4.5 State Feedback Including a Kalman Filter 
(Discrete-time System) 

The block diagram of such a control system is shown in Fig. 4.8. The Kalman fil- 
ter plays the same role as an observer, seen in Chap. 2, with the additional prop- 
erty of filtering noises, in the case of noisy plants. 



White noise V, 



y r ,t 



^QirH* 




State feedback 
Fig. 4.8 Discrete-time state-feedback control system with Kalman filter in the loop. 



4.5.1 Closed-loop Transfer Function 



Hypotheses: in this section, we will assume that 

1 . the plant is linear and time -invariant (constant matrices), with matrix D = ; 

2. the noises vanish; 

3. the initial conditions of both the plant and the filter are all zero. 

Under these hypotheses, (4.88) yields 
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**l*-i = *(I KC)xn| t _ 2 + ru t _! + OKy t _, , 
hence, with y k _ x = Cx t _] , 

In the control law u^ =— Lx A , x t will be replaced either by x t | t _j or by 

x t | t . Let us make the first choice, the most common in practice since it allows 
computing u k in advance, during the interval of application of the present control 
u t _! , from the values available at the present step, k — l. 

By application of the z-transform, with the notations 2 ^tlt-i = ^~(z) , 

Jj[x t ] = X(z) and Jj[u t ] = U(z) , we can write: 

X-(z) = z- 1 0(I-KC)X-(z) + z" 1 rU(z) + z- 1 OKCX(z). (4.93) 

On the other hand, by applying the z-transform to (4.79) at V k = , 

x(z) = (zi-o)- 1 ru(z) = z- 1 (i-z _I o)- 1 ru(z). 

Combining these two equations yields 

[l-z^ 1 0(I-KC)]x-(z) = z- 1 rU(z) + z- 1 OKCz- 1 (I-z- 1 0)" 1 rU(z) 
= z- 1 [H-4»KCz- 1 (I-z _1 <l>r 1 ]rU(z) 
= z- 1 [l-z- 1 + OKCz- 1 ](I~z- 1 0)" 1 rU(z) 

= z- 1 [i-z- 1 o(i-KC)](i-z- 1 or 1 ru(z) , 

from where results successively that 

X"(z) = z _1 (I -z" 1 0)- 1 rU(z) = X(z) , (4.94) 



The prediction calculated by the Kalman filter "sticks" to the true state. The 
same should hold for the estimate x k \ k . Let us verify it. According to (4.81) or 
(4.54), 
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*k\k = *k\k-l + K (y* _ C^tli-l) • 
Hence, from above: 

X k\k = X (tli-1 + K(C,X A — CXj|j_j ) = Xj.Ij._j = Xj . 

Conclusion. The closed-loop transfer function is the same as if the loop were 
closed without any Kalman filter included. 

This sends back to the discussion in Sect. 2.5.5, concerning the behavior of a 
closed-loop system including an observer in the loop with respect to reference 
changes, when the plant and observer initial conditions are identical, or once the 
estimated state has converged to the true state. This should not constitute a sur- 
prise, if one remembers that a Kalman filter is nothing else but an observer for 
noisy systems (see Sect. 4.3.1.5). 

The gain compensation matrix will thus have the same value as calculated 
without filter. Let us verify this. 



4.5.2 Calculation of the Gain Compensation Matrix 

By inspection of Fig. 4.8 or by rewriting (4.93) with u k = — Lxj.ij._j + My r k and 
then applying the z-transform, we obtain: 

X-(z) = z- 1 [0(I-KC)-rL]X~(z) + z- 1 OKCX(z) + z- 1 rMY r (z), 

where __ fy k 1 = Y r (z) . Multiplying now both sides by z yields: 

(zl O + OKC + T L) X" (z) = OKCX(z) + TM Y r (z) . 
Taking into account (4.94), this equation writes also: 

(zl - <D + OKC + TL) X(z) = OKCX(z) + ITVIY r (z) , 

(zl - O + T L) X(z) = rM Y r (z) , 
hence, finally: 

Y(z) = CX(z) = C(zI-0+rL)-' FMY r (z) . (4.95) 

The requirement Y(z) = Y r (z) in response to a constant reference y r yields thus, 
by letting z = 1 in (4.95), the following relation for M: 
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M = fc(i-o+rL)- 1 r]~ 1 . 



This expression is exactly the same as in the case of a direct state feedback, with- 
out filter in the loop, as given by (1.16). 



4.6 Principle of Duality 



4. 6.1 Already Encountered Dualities 

Let us recall the very first duality identified in Appendix A, between the control- 
lability and the observability canonical forms of the state equations. The equations 
of either one of these forms are obtained from the other by simple symmetry of the 
triplet A, B and C ( O, T , and C) with respect to the main diagonal of A ( O ): 



A(*)<-> 


A t (O t ) 


B(D~ 


C T 


C^ 


B T (r T ) . 



Later we have remarked that there exists a duality of the same nature between a 
state feedback and an observer design, thus between a control and an estimation 
problem. 



4. 6.2 Duality between Filtering and Control 

Now, as mentioned previously, the observer is a particular case of the optimal fil- 
tering. It is therefore not surprising to recognize this duality between the optimal 
linear filtering and the optimal control theories. These two theories have indeed 
many dual aspects, among which the fact that both require the solving of a matrix 
Riccati differential, or difference, equation. The essentials of these duality proper- 
ties are grouped in Table 4.1 for the continuous case, and in Table 4.2 for the dis- 
crete case. 

In practice, this means that the design of an optimal filter and that of an optimal 
controller can be made with the same numerical computation programs. This has 
been used extensively in the creation of the accompanying software, MMCE.m. 
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Table 4.1 Extended duality between filtering and control, for the continuous case. 



Optimal filtering 






Optimal control 


Q 






Q T 


R 






R T 


K 






L T 


I 






P T 


t 






-t 


jX(0 = A(f)X(f) + E(OV(0 






±(t) = A(t)x(t) + B(t)u(t) 


[y(0 = c(0X(0+w(0 






[y(t) = C(t)x(t) 


Optimal estimate: 




Optimal state feedback: 


l(t) = A(t)x(t) + K(t)[y(t)-C(t)x 


(01 




u(t) = -L(t)x(t) 


with: 




with: 




K(t) = -L(t)C T (t)R-\t), 






L(0 = R"'(0B T (0P(0, 


where Z(7) is solution of the (differential) 




where 


?(t) is solution of the (differential) 


Riccati equation 




Riccati 


equation 



X(t) = A(t)-L(t) + -L(t)A l (t) 

-I.(t)C J (t)R- 1 (t)C(t)I.(t) 
+E(0Q(0E T (0 



• P(0 = P(0A(0 + A T (0P(0 

-P(0B(0R"'(0B T (0P(0 
+Q(0 



For the discrete case, due to limited space and also to avoid unnecessary dupli- 
cations, these properties are limited to the Kalman and state-feedback gain matri- 
ces, and to the Riccati equations. 

Table 4.2 Some duality properties between filtering and control, for the discrete case. 



K, 



'klk- 



Optimal filtering 

-i C k( C k^k\k-l C k + R t) 



: ^k\k C k^k 



■•k+iik 



: ¥nt-A -^S t|i _ I Ci(C t L t|t _ 1 Ci+R t r 1 C t Z t|t _ 1 «i t 1 +E t Q t Ei 



Optimal control 



o; p t+1 o k - o k l p t+1 r t (R t + r l k v k+l r k r 1 rjp i+1 o k + q 
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4.6.3 Principle of Duality in Mathematics 

The following definition gives a broader scope to the above identified duality. 
Principle of duality. Given the following system in state space representation: 

jx(0 = A(0x(0 + B(0u(0 
the system defined as follows: 

f-4(0 = A T (0$(0+c T (0v(0 



i|(0 = B 1 (^(0 



(II) 



and which is obtained from the initial system by reversing the time progress, by 
swapping the input and output matrices and by transposing the A, B and C, is 
called dual system of the first one. 

It is worthwhile to note that these two last equations amount to replacing the trip- 
let A, B and C by its symmetrical triplet with respect to the main diagonal of A. 
The estimation and the filtering of the initial system (I) correspond to the control 
of the dual system (II), and vice versa. 

Remark 4.13. In differential calculus, the system (II) is also called adjoint system 
of system (I). 

It is easily verified that the application of this principle of duality, due to Kal- 
man (1960), permits retrieving all the dual results stated above. By giving the pos- 
sibility to transpose a number of results from the optimal control theory to the do- 
main of estimation and optimal filtering, he accelerated considerably the 
development of this second theory, historically more recent. 

Let us mention, by curiosity, that the use of the mathematical duality principle 
yields directly the adjoint equation of a linear system, without resorting to (3.13). 



4.6.4 Duality of the Transition Matrices 

The previous symmetry rule extends also to the transition matrices of the two sys- 
tems. 

Denote indeed by *F the transition matrix of the adjoint system (II). It will be 
solution of the homogeneous differential equation of the corresponding system 
(see Sect. A. 5), thus of 



V(t,t ) = -A T (t)V(t,t ). 



242 4 Noisy Systems - Optimal Linear Filtering 

By grouping all the terms to the left side of this equation and premultiplying by 
the transpose of the transition matrix &(t,t ) of the initial system, the following 

holds: 

® T (tJ )V(t,t ) + ® T (t,t )A T (t)V(t,t ) = 0. (4.96) 

Since 

®(t,t Q ) = A(t)<l>(t,t Q ), 

it results, by substitution in (4.96), that 



i.e. 



or: 



d_ 
dt 



<S> T (t,t )V(t,t ) =0, 



O t (t, t ) "¥(t, t ) = constant . 

Since we have O T (t ,t ) = *¥(t ,t ) — I at t — 1 , this implies that the matrix con- 
stant in the right side is also equal to I: 

® J (t,t )*¥(t,t ) = I. (4.97) 

Consequently, 

i.e.: 

*¥(t,t ) = ® J (t ,t) (4.98) 

The transition matrices of the two systems are thus also dual to each other, ac- 
cording to the previous principle of duality: by transposition and time reversal. 
Furthermore, with (4.97) it is possible to write: 

In other words, the dot product of the state vectors of the two dual systems is 
constant. 
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4.7 Correspondence of Notations in this Book with 
those of MATLAB® 



Equations (4.88) (a) and (b), associated with (4.89), once the time dependence in- 
dices of the parameter matrices have been suppressed, are those of the state repre- 
sentation used by MATLAB® in its script kalman.m to model a Kalman filter in 
terms of an LTI model 1 . The inputs to this linear system are respectively the input 
u k and the measured output y^ of the plant whose state is to be estimated, while 

the filter output is constituted by the output y and the state x k \ k , both esti- 
mated after the measurement. 

This script solves the stationary Kalman filter equations. It calculates also the 
steady-state estimation error variance before and after measurement 5T and X + , 
according to (4.75) and (4.77). The correspondence between the MATLAB® no- 
tations and those of this book is summarized in Table 4.3, for the discrete systems. 

Table 4.3 Correspondence of notations with MATLAB®. 



MATLAB® 


This book 


A 


o 


B 


r 


w 


v 


V 


w 


G 


E 


QN 


Q 


RN 


R 



x \n \n\ 



Ul/t 



x[« + l|«] x t+1 , t 

M K 

L OK 



p ^k\k-\ ° r s " 



s i|i ° r £+ 



The script file kalman.m is in the Control Systems Toolbox. On-line help is available by help 



kalman and help dkalman for the continuous and discrete Kalman filters. 
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4.8 Extended Kalman Filter 

Though this book deals almost exclusively with linear systems, a brief exception 
will be made in this section. There is indeed an interesting extension of the Kal- 
man filter to nonlinear systems, named Extended Kalman Filter (EKF). It will be 
presented briefly here, only in the discrete case which represents the majority of 
applications. Some details about this subject can be found in [Sim06], [Rib04]. 



4.8.1 Basic Idea 

The basic idea, due to Stanley Schmidt, is the following. To reuse as much as pos- 
sible the previous linear formulations, the approach will necessarily consist in lin- 
earizing the nonlinear system to be estimated, around a nominal state trajectory. 
But how to determine the nominal trajectory? The system being nonlinear, this 
might be a non trivial issue. The idea was thus to use the Kalman filter estimate of 
the system state as the nominal trajectory. The extended Kalman filter operates 
thus in sort of a bootstrap way: at each step, the nonlinear system is linearized 
around the Kalman filter estimate, which becomes the new nominal state, and the 
Kalman filter estimate of the next step is based on this linearized system. 



4.8.2 Problem Presentation 

Assume a nonlinear system is described by the following state equations: 

1 Y k =g k (X k ,u k ,W k ) 

where the process noise, V k , and measurement noise, W^ , are supposed to be 

distinct white noises, thus not correlated with each other. The hypotheses are thus 
the same as in the linear situation (see (4.44)), and are summarized here again: 



E{VJ = 0, Cov{v t ,y / } = Q i 8 w> 
E{WJ = 0, Cov{w i ,W / } = R t 5 w> 

Cov{V t ,X i } = 0, Vj<k, 
Cov{W t ,X,} = 0, \/j,k. 



4.8 Extended Kalman Filter 

We will denote, as for the linear Kalman filter, by 
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• Xfc|£_i , the optimal estimate of the state X k elaborated before the measurement 
y k , or prediction of the state X k ; 

• x t | t , the optimal estimate of X k elaborated after the measurement y k , or es- 
timation of the state X t . 

To derive the equations of the discrete extended Kalman filter, we will proceed in 
two phases: linearization of the nonlinear equations, and then application of a lin- 
ear Kalman filter. 



4.8.3 Linearization of the Nonlinear Dynamic Equations 

The evolution equation is linearized first, by Taylor series expansion in the vicin- 
ity of X k = x t | t and of V^ = , with our notations of Sect. B.3.2 (Appendix B): 



X/t+i — f k (x k \ k ,u k ,0)- 



df k (x k ,u k ,v k ) 



dX T k 



p^k ~ x k\k) 



df k (x k ,u k ,v k ) 



d\l 



V,. 



(4.99) 



Xt-Xj |i 



Let us introduce the following simplifying notations, the derivative of a vector 
with respect to a vector being a matrix (see (B. 12), Appendix B): 



df k (X k ,u k ,V k ) 



dXl 



and L **<X*.»*.V t ) 



Xi-X t |i 



dvl 



Xt-$ty 



Note that these two matrices are deterministic. (4.99) is then rewritten as follows: 



f t (x, | ,,u„0)+F,(X,-x, | ,) + L,V, 



St+l — l k\*k\k>"k 



^X,+[f t (x t|;t ,u t ,0)-F;x, |t |+-L,V s . 



By defining the following quantity, also deterministic, 



s t — f/t(xt|t,u t ,0)-F t x t | t , 



we obtain finally: 
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H+l — A k A k 



FX+S.+LV, 



t T /t • 



(4.100) 



The measurement equation is linearized next by Taylor series expansion in the vi- 
cinity of X k — x /t | t _ 1 and of W k = : 



Y i=g/t^tlt-i» u i»°)- 



3g t (X*,u t ,W t ) 



axj 



(Xjt —*-k\k-l) 



+ 



^k—^k\k-l 

d gk (x k ,u k ,w k ) 



dWj 



W k . (4.101) 



The following deterministic matrices are defined here, for simplification: 



dz k (x k ,u k ,w k ) 



axl 



and M k = 



dg k (X k ,u k ,W k ) 



A-fc— X i- 



9W A T 



^*-i— X £|£-l 



(4.101) rewrites then 



Xfc=g i (x i | t _„u i ,0) + G t (X 4 -i i | t _ 1 ) + M i W t 

= G t X t +fg i (i i | i _ 1 ,u iJ 0)-G t i t | i _ 1 ] + M t W t . 



By defining the deterministic quantity 



r i=gi(Xi| i -i,u i ,0)-G i x 



tli-l ' 



we obtain finally: 



Y t =G t X t +r t +M t W t . 



(4.102) 



4.8.4 Derivation of the Discrete EKF Equations 



We have now a plant described by a linear evolution equation and by a measure- 
ment equation, which is also linear. It is thus possible to apply a linear Kalman fil- 
ter to this plant to estimate its state. 

We will use the equations of the linear Kalman filter, with a deterministic input to 
the plant, seen in Sect. 4.3.4. The discrete linear plant being described by equa- 
tions (4.79), recalled here for commodity, 
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x t+1 = o,x, + r t u t + E,v t 

Y,=C,X,+D t u t + W, 

the discrete linear Kalman filter obeys the following equations, recalled also from 
Sect. 4.3.4: 

• Estimation (or update) phase (equations (4.81), (4.82), (4.55) and (4.56)): 

%\k = x*i/t-i + K /t(y* _c Ai*-i -D /fc u /t) > 

y k \ k = c k*k\k+v k u k , 

**-k = ^t|t-i*--t(C t Zt|t_i C t + R t ) =E t | /t C /t R t , 

Var|X t | t j = Z t | A =Z t | t _, — Z t | t _jC t (C A Z^^C^ + R t ) C^Z^^ 
= (I-K t C t )Z t|t _, 

• Prediction (or propagation) phase (equations (4.83) and (4.59)): 

x k+i\k = % x ti<t + r*u t , 

Z t+1| ,=0,I^< + E t Q t ET. 
By making in these equations the following substitutions: 

^iUj = s t , 

C /c = G t > 



and 



D *U* = r t . 



and by noting that, since the white noise W t is centered, the noise Z k = M t W t 
involved in (4.102) is also centered, and that its variance amounts to 

Var{Zj = E{M,W,(M t W,) T } = M,E{W ;t W/}MT=M,R t MT, 
the following equations result for the discrete extended Kalman filter: 
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Estimation (or update) phase: 

r t =S*(x*lt-i,u*,0)-G i x t | t _ 1 
5*lt =i*lt-i+K i (y t -G t x i | t _ 1 -r i ) (4.103) 

= x t |i_i +K t [y t -gt^tit-i.Ut.O^iti^! + K t (y t -J^) 

K, = I t|t _, Gj(G t Z t|i _, Gj + M.R.Mj)" 1 - Z t|t Gj (M^M*)" 1 (4-104) 

Var{X t|i } = E i|i 

= Z^ - 2^., Gj(G t Z^., Gj + M.R.Mjr'G, £,„_, ( 4 - 105 ) 
= (1-^0^^ <*10Q 

Prediction (or propagation) phase: 

i- t+ il*=f*fin*>u*>0) (4 - 107) 



■"t+ii* 



F t Z t|t F T + L 4 Q, L T t , Z 0M given (4.108) 



Remark 4. 14. Equation (4. 103) lets us understand why the measurement equation 
(4.101) has been linearized around X t =x A .| t _ 1 and not X k = x k \ k . The reason is 
that the correction term in the update phase involves the difference between the 
new measurement, y k , and its prediction, y.,,_. =g J: (x i | i _ 1 ,u i ,0) . If (4.101) had 

been linearized around X k =\ k ^ k , leading to r k = g k (x k \ k ,u k ,0) — G k x k \ k , then 
in the update phase x k \ k would be used to calculate the value of . . . x k \ k itself! 



4.8.5 Condensed Form of the Estimation and Prediction 
Equations 

• Estimator filter, obtained by eliminating \ k \ k _ 1 between (4.103) and (4.107): 

Xt|i-=ft-i(xt_ 1 | t _i,u t _ 1 ,0) + K t [y t -G t f t _ 1 (x t _ 1 | t _ 1 ,u t _ 1 ,0)-r /t ] .(4.109) 

• Predictor filter, obtained by eliminating x k \ k between the same equations: 

^+ili= f /tfxiit-i + K t (y i -G 4 x i | i _ 1 -r i ),u Jt ,0l . (4.110) 
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4.9 Solved Exercises 



Exercise 4.1 Discrete-time Kalman Filtering of an Air- 
flow Heater 

Let us consider again the air-flow heater described in Exercise 3.6. As in Chap. 3, 
the present exercise will be solved entirely on a simulation model of this device. 
The plant being intrinsically noisy, it will be simulated by incorporating two 
pseudo-white noise generators in the Simulink® diagram, one for the measure- 
ment noise, the other one for the process noise. 

a) Design by means of the program MMCE.m, for the discrete model of the 
process trainer PT326, without integral action, a state feedback by pole 
placement, by imposing to the closed loop the continuous values 
S t = — 10, i = 1,2 . Design then a Kalman filter, by choosing for the covari- 

ance matrices of the process noise and of the measurement noise the fol- 
lowing values: Q — 100 ; R = 1 . 

Simulate the responses by connecting alternatively the two noise generators 
to the diagram by means of the available switches, and by letting the Kal- 
man filter operate in^ree wheel in order to see better its effect, the plant 
remaining fed back from its real (simulated) state vector. 

b) Repeat the Kalman filter design with: Q = l; R = \00 . 

Repeat the simulations with this time the filter included in the loop. Com- 
pare the results. 

c) In order to illustrate the control-estimation duality, calculate instead of the 
Kalman filter an identity observer, and design it by quadratic criterion, by 
choosing the following weighting matrices, and then compare the results 
with those of the filter of question (a): 

Q = E-100-E T , where E = T; R = l 

Solution: 

(a) The program MMCE.m yields £ T = (-0.469 0.242) , M = 2.465 , and, for 
the Kalman filter: 

' 0.848 ) 
10.891 



steady state Kalman gain: K = 
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• matrices of the filter state equation (4.88)(a), with D k = , in the form of an 
identity observer, according to the equations (2.9) and (2.10): 



X/t+i — F x A - 



Tu.+Gy 



k ' 



G = OK 



(1.055) 
7.589 



F = <D GC 



f— 0.965 0.090) 
-9.027 0.809 



Perform then a simulation, where only the process noise generator is connected to 
the plant. The covariance matrices chosen in this question indicate indeed that the 
measurements are only slightly disturbed, by comparison with the state which is it 
on the contrary strongly. 

The left side of Fig. 4.9 shows the response of the real states to a square wave 
reference, whereas its right side shows the response of the filtered states, the Kal- 
man filter remaining out of the control loop. The comparison of these plots illus- 
trates clearly the filtering effect. 
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Fig. 4.9 Closed-loop response to a square wave, (a) of the real states, (b) of the filtered states, 
the Kalman filter being out of the loop. 
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If now the measurement noise generator is connected and the other one discon- 
nected, there is no longer any filtering effect. Even worse: since the state is not 
disturbed at all in this case, but the measurement is, and since we "made the filter 
believe" that the measurement is very reliable, it puts great confidence on it to re- 
construct the state and reproduces on the latter the full measurement noise! 

(b) The following matrices are obtained here: 



K 



0.0019) 
0.0160 



G=OK 



0.0016) 
0.0102 



F = <& GC = 



0.088 0.090) 
-1.448 0.809 



A big difference with the previous case is noticed here, as to the value of the 
Kalman gain K: since the filter has been informed this time that the measurement 
noise is significantly stronger than the process noise (or model uncertainty), it will 
give only very little weight to the innovation furnished at each step by the meas- 
urement and will rely primarily on the plant model to reconstruct the state. This 
situation is illustrated strikingly in Fig. 4.10. 
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Fig. 4.10 Same as Fig. 4.9, but with filter weightings of question (b): 
(a) (left): real states; (b) (right): filtered states. 
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Here an almost perfect state variable reconstruction takes place, be it in the 
presence of measurement noise or of process noise. The filter operates here even 
better, included in the loop, since it receives at its input the control signal which 
has already been filtered at the previous step, which is not the case when it oper- 
ates in free wheel. 

(c) The matrices found for the observer are: 



-0.965 0.090) 
-9.027 0.809 



(1.0551 
7.589 



which are exactly the values found in question (a). 

This is a confirmation of the duality mentioned in this chapter: a Kalman filter 
is nothing else but the dual of an optimal controller. The dlqr algorithm solving a 
Riccati equation which is dual of that of an optimal filter should therefore yield 
the same values if it is given the transposed matrices, as discussed in Sect. 4.6. 

The particular choice of the Q matrix results from the fact that the Riccati 
equation solved by the kalman. m module is of the form (4.63), where the term re- 
lated to that matrix appears in the form E^Q^E^ , whereas it is simply equal to Q 
in the Riccati equation (3.80) of the optimal control solved by dlqr.m. 



Exercise 4.2 Control of a Magnetic Tape Drive 

The plant studied in this exercise is sketched in Fig. 4. II. 2 

I ► x l I ► x 3 I ► x 2 




A 




Fig. 4.11 Magnetic tape drive. 

Since the magnetic tape is made of a flexible material, the objective of the con- 
trol device, composed of two DC motors placed at each end of the tape and con- 
trolled separately, is to position the tape over the read head while keeping its ten- 
sion Tat some specified value. With the following parameters, 



' Exercise inspired from [FrPW97] . 
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J = moment of inertia of a motor/capstan assembly = 1.6 x 10~ 4 kg • m 2 , 

r = capstan radius = 0.1 m , 

K m = torque constant of the motors = 0.064 N • m • A~ , 

k= coefficient of elasticity (spring) of the tape =2000Nm~ , 
B = damping coefficient of the tape = 3.2 N • m _1 • s , 

the equations of motion of this system write: 



\j0 l =Ju} 1 =-Tr + KJ l 



J9-, — Jui', 



-Tr + KL 



with : T = k(x 2 —x l ) + B(x 2 — x x ) . 



The resulting continuous-time state model is the following: 

( -100 

100 

1.25 -1.25 -0.2 -0.2 

1.25 -1.25 -0.2 -0.2 



*1 


/ 


x 2 
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W] 




Cj 2 





x x 




' 





x 2 
u> x 


+ 



0.4 






[lj 2 




. 


0.4 



l 2) 



where 

• x u x 2 = tape position at the capstans (mm); 

• x 3 = (xj + x 2 )/2 = tape position over read head (mm); 

• uj x ,bJ 2 = angular velocities of the motor/capstan assemblies (rad/s) 

The output quantities which need to be regulated, x 3 and T, are given by: 



(0.5 0.5 

2 -2 0.32 0.32 



The specifications are the fallowings: 

• a step change of 1 mm of the position x 3 of the tape above the read head 
should occur with a 1% settling time less or equal to 400 ms and an overshoot 
less than 10%, the tape being at rest before and after the position change; 

• the tape tension T should be controlled to 2 N, while staying always between 
and 4 N; 

• the armature current of each motor must not exceed 4.9 A in steady state and 
30 A during the transients. 
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This problem will be solved in discrete time, the plant being sampled at 
T = 0.01 s . 



a) Suppose first the state completely accessible, and calculate an optimal 
state-feedback control law, with output weighting and the following matri- 
ces: Q = R = I 2 . 

Determine the equivalent continuous-time poles (s t , / = 1,...,4) of the 
closed loop. 

Record by simulation the step responses of x 3 and of T, as well as the cor- 
responding control signals. 

b) Having noticed that the specifications are not fulfilled, repeat he previous 
question, with the following choice: Q = 100 x I 2 and R = I 2 . 

c) In order to get rid of eventual variations of plant parameters, a partial inte- 
gral action is now introduced on the tension, which is the most sensitive 
quantity to these variations in this problem. Repeat the design of an optimal 

flOO 0) 

state feedback, with output weighting and Q = 



100 
1 



and R = I 2 , 



the increased size of Q resulting from the fact that the integrator output 
state variable is also taken into account in the weighting. The choice of 
q 33 = 1 stems from the necessity not to accelerate excessively this added 

mode, in order to keep the control signals below the allowed limits, as will 
be checked in simulation. 

d) Calculate an optimal estimator (Kalman filter) allowing the state recon- 
struction from the two only available measurements, knowing that the 
mean quadratic error of the position measurement amounts to an RMS 
value of 0.02 mm, while that of the tension measurement has an RMS 
value of 0.01 N. It will be assumed further that the noises and modeling er- 
rors which affect the motors are taken into account arbitrarily by the fol- 
lowing covariance matrix of the process noise: Q = 10~ xl 2 . 

Solution: 

(a) Run the MMCE.m program by choosing the plant magnetic tape drive, 
discrete model, and design a state feedback, without integral action, by quad- 
ratic criterion (LQC), with output weighting, and Q = R = I 2 . The following 

feedback gain and feedforward matrices result: 
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-0.534 -0.147 17.32 -1.136) 
0.147 0.534 -1.136 17.32 
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M: 



-0.681 1.659) 
0.681 1.659 



The program yields the closed-loop poles in the z-plane. Their continuous-time 
equivalent is given by 



*,.=(l/7;)ln(z,.) 



s,. =-3.76 ±j 3.76 s" 1 and -3.50 ±7 16.19 s" 1 . 



The simulated step responses show that the specifications are not fulfilled: the 
1% settling time of the position in response to a 1 mm reference step reaches 1.2 s. 
The control currents, on the contrary, are about 3 A in steady state, thus below the 
allowed nominal value. 

(b) Greater weights are taken now for the trajectory, so as to accelerate the 
transient response by diminishing the deviation from the position step: this justi- 
fies the choice of the larger ratio between the Q and the R matrices proposed in 
the problem statement in this question. The control law becomes here: 



— 11.4 5.47 75.40 19.38) 
5.47 11.74 19.38 75.40 



M = 



'-6.28 5.87) 
6.28 5.87 



and the closed-loop s-poles are now: 



s, =-11.9 ±7 11.9 s" 1 and - 21.3 ±7 26.5 s" 1 . 



The state-feedback gain is significantly larger than in the previous question, 
which provides noticeably faster closed-loop poles. This is confirmed by the step 
responses in Fig. 4.12. 
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Fig. 4.12 Closed-loop step responses, without integral action: (a) outputs, (b) controls. 



256 



4 Noisy Systems - Optimal Linear Filtering 



(c) After the model selection and the design of a state feedback, one chooses 
here with integral action and, for the Row-vector with indices of y- 
components to feed back through integrator(s): 2 or [2], since it is to the out- 
put y 2 — T that we want to apply partial integral action. This creates the selection 

matrix S, = (0 1) (Sect. 1.8.3). The obtained matrices are the following: 



Li 



[-16.31 10.03 84.43 28.40 
-10.03 16.31 28.40 84.43 



-0.547' 


; M = 


-6.28 


8.15' 


-0.547 




, 6.28 


8.15 



Note that here S c = I 2 , since p = q which has allowed calculating M by the 
relation (1.16). 

The step responses are given in Fig. 4.13. 
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Fig. 4.13 Closed-loop responses to reference and load disturbance steps; partial 
integral action is applied to the tension: (a) outputs, (b) controls. 

The control currents remain below 4.9 A in steady state and below 30 A during 
the transients. It is this last limit which would have been exceeded if, instead of 
<7 33 = 1 , we would have chosen <7 33 = 100 . 

The figure shows, as in the previous case, that unit static gains are guaranteed 
between the two references and the corresponding outputs. Furthermore a constant 

load disturbance, equal to (2 4) and applied through the matrix E — E m to the 

two inputs at t — 0.5 s , is rejected in steady state on the output y 2 = T , but not on 

y 1 = x 3 , as expected. By repeating these simulations, the reader can verify that 

this is also the case for a constant measurement disturbance. 

The step response overshoot of about 30% on the tension results from the zero 
added to the closed loop by the anticipation term M, namely, in the z-plane: 
z 3 = 0.9328 . Since this zero is significantly "slower" than the one which the 

closed-loop system had without integral term and which was already an open-loop 
zero, z 2 =—0.5152 , it is not surprising that it augments strongly the overshoot. 
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As indicated in the discussion of Sect. 1.8.3, its effect can be reduced by diminish- 
ing the gain of this term by means of a multiplicative coefficient p < 1 . With 
0.5 x M , this overshoot is lowered to 15%, but at the cost of a 50% reduction of 
the static gain on the output y l , which of course does not fulfill anymore the 
specifications. 

(d) Back to the main menu choose the synthesis of a Kalman filter, with the 
following noise covariance matrices. For R , select according to the data in the 
problem statement and by application of (4.92): 



R 



0.02' 







o o.Or 



0.0004 
0.0001 



The filter is calculated obviously for the initial discrete-time plant, without inte- 
grator: it would make no sense to estimate the quantity delivered at the output of 
an integrator. The resulting Kalman gain is: 



K 



0.0518 


-0.0159 


0.0518 


0.0159 


-0.0014 


0.0006 


0.0014 


0.0006 



In the simulation diagram two uncorrelated measurement noises, with respec- 
tive variances of 0.02 2 mm 2 and of 0.0 1 2 N 2 , are applied to the measured out- 
puts of x 3 and of T made available in the diagram by the block C m . The obtained 
step responses are plotted in Fig. 4.14. 
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Fig. 4.14 Step responses of the two outputs in the presence of measurement noise. 
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The noise generator of the diagram is in fact a pseudo-white noise generator, 
whose power spectral density is chosen equal to (0.02 2 0.01 2 )xr 5 , in order to 

account for the relation between this parameter and the covariance of the signal 
generated by this type of block. 
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5.1 Problem Description. Discrete-time LQG Control 

This chapter deals with the optimal control of a noisy linear system, the state of 
which is not entirely available, i.e., which requires a state reconstructor in the con- 
trol loop. Since the system is submitted to random influences, a filter, e.g. an op- 
timal filter such as the Kalman filter, will be used. 

In the rather frequent case where the stochastic processes applied to the plant 
are Gaussian, such a control is called LQG, for Linear Quadratic Gaussian. 

Stated like this, the problem is similar in its structure to that of a state feedback 
control of a deterministic system, using an observer in the loop to reconstruct part 
or all of its state vector. 

Thus the following question arises: does the separation theorem apply here al- 
so, i.e., can we dissociate the synthesis of the LQG optimal control law from that 
of the Kalman filter, as we could do it in the deterministic case for the synthesis of 
a controller and an observer introduced in the loop? 

Let us treat here the case of discrete-time systems. The plant is a noisy, time- 
varying, linear discrete system, described thus by equation (4.79) of Sect. 4.3.4, 
where however the assumption D A . = will be made to shorten the equations: 

The other hypotheses are those of Sect. 4.3.1.1: the random processes V t and 
W k are white noises, uncorrected with each other and with the past and present 
states of the system, and having respective covariance matrices Q A . and R k . The 
plant initial state X is random and is described by: 

EJXoj^x.o; Var{X } = S . 
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Let be given, furthermore, a quadratic cost functional of the type introduced by 
equation (3.72) for time-invariant discrete linear systems, but whose formulation 
will be extended here to the case of time- varying systems, and of weighting matri- 
ces of the two quadratic forms, Q k and R , eventually also time dependent: 



J = E\^X J N SX N +^(XlQ k X k +ulR k u k ) . (5.2) 



Note that the state vector involved in this criterion is random, which suggests to 
take the mathematical expectation of the expression which had been used for J in 

(3.72), so as to obtain a deterministic cost functional, denoted here by J . 

The objective is to determine a control law u* k function of the measurements 

y , y !,..., y k , which minimizes the cost functional J . This approach is called 
stochastic optimization of the control. 



5.2 Stochastic Separation Theorem 

The hypotheses are: 

1 . the plant is linear; 

2. the cost functional is quadratic; 

3. the added noises V k and W t are white noises, not correlated with each other, 
V A being uncorrected with all past and present states and W k with all states. 

Theorem 5.1 (separation theorem). The optimal stochastic control of the system 
(5.1), with the cost functional (5.2), is obtained by taking the optimal control law 
u k =— Lj x k given by (3.78) and (3.79), calculated for the corresponding deter- 
ministic system without noises, 

X *+l = <*>k X A- + I> t , 

with the cost functional 

1 1 N ~ l 

J = -x T N Sx N +-Y J (x T kQkXk+»k' R k Uk), 

*■ *■ k = 

and by replacing in this control law the plant state x by its optimal estimate 
x k i k _ l , yielded by a Kalman filter: 
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X k\k-l — ®k-l X k-l\k-2 +^ , i-lK t _ 1 (y t _| — C k _ l x k _ l \ k _ 2 ) , 
where 

**-k = ^k\k-l Ck(Ck ^k\k-\ *-"* +™t) > 

I i+1| ^O i I i|H O i T -O i 2 i|i _ l Cj(C i Z t|i _ l C[ + R i )- 1 C i Z t|H O t T 



"*Ar+l|Jt — v t^t|*:-l v t , *V*|*-l v "*A*-*'"i|i(r-l*-it ~ iv it^ " k -jt-*'*:|*r— l^Tt 



+E t QX . 



7%e sought control law is thus, according to (3. 79) where eventual time dependen- 
cies of the matrices have been reintroduced: 

u k = — L t x A .| A _ ] = — (R t + r k r k+1 r k ) T k P k+l <i> k x k \ k _ l . 

Such a synthesis is therefore decomposed in two independent problems: 

• the optimization of the control of the deterministic, not noisy plant; 

• the optimization of the estimation of the state X from the measurements y. 

Partial proof of the theorem. By using the error estimation before measurement, 
X t | t _! = X t — 5fc|fc_i , introduced in Sect. 4.3.1.2, we can replace X k in the quad- 
ratic form defined by Q k involved in the expression of J by 

x t =%\k-i + x*i*-i • This y ields 

T ^T ~ T ^ ~ 

X *Q* X * = ( x <ti<t-i + x tin)Qt ( x k\k-i + x Ai/t-i) 

^T ^ ~T ~ ^T ~ 

= X k\k-l Qk X k\k-1 + X £|/t-l Q/t X t|t-1 + ^ x k\k-\ Qk X ilA:-l • 

Since b T a = tr(ab T ) , according to (B.2), the last term of this sum writes also: 

~ ^ T 

2tr(Q /fc X t | t _! • x t | t _!). 



Now 



T 



E ti"(Qt x ti,t-r x *u-i) =Qt tr E \ x k\k-i • x Ait-i 



T 



= 0. 



since the estimation error is orthogonal to the estimate. The corresponding term 
will thus disappear from (5.2). One would obviously prove that this is also the 
case of the term associated with the quadratic form defined by S , so that (5.2) be- 
comes: 
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y = E 



1 ^T f . . 1 ^1 _T T 

1 k = 



1 ~T ~ 1^~T | 

7yN\N-\ ™X N \ N _ X +T / j^k\k-\ Q/t ^k\k-\ [ • 
1 Z k=0 j 

The first expectation in this expression yields the part which will be minimized by 
a suitable choice of the sequence of control signals {u t }, since x^^ depends on 
the sequence u , ... ,u k _ { . The second expectation will be minimized on the con- 
trary when the estimate of X k will be optimal. 

This proves, at least heuristically, the separation of the posed problem into a 
deterministic optimal control problem and a stochastic optimal filtering problem. 
The controller and the filter can therefore be determined completely separately. 



5.3 Continuous-time Systems 

The situation is of course very similar to that of the discrete -time systems. Con- 
sider a time-varying linear system, with D(t) = , as given by (4.80): 

X(0 = A(0X(0 + B(0u(0 + E(0V(0 

Y(0 = C(0X(0 + W(0 

The random processes V(t) and W(7) are white noises, not correlated with each 
other nor with X(t) , and having respective power spectral densities Q(t) and 
R(7) . The plant initial state X is random and described by: 

E{X } = k i X0 ; Var{X } = S . 

The stochastic optimization of the control will have to minimize the following cost 
functional: 

J = E \-X r (t f )SX(t f ) + - f f [x r (t)Q(t)X(t) + u T (t)R(t)u(t)\dt\. 

Assume that an optimal control law has been designed for the plant without 
noise, thus of the form 

u(t) = -L(t)x(t), 



5.3 Continuous-time Systems 
with 
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L(0 = R-'(0B T (0P(0, 



where P(t) is solution of the Riccati differential equation. 

Since the state is not available, let us replace it in this control law by its optimal 
estimate x(t) : 

u(0 = -L(0x(0, 
and introduce the estimation error, X(7) = X(t) — x(t) : 



u(0 = -L(0[x(0-x(o|- 



The closed-loop state differential equation becomes: 



X(t) = A(t)X(t) - B(?)L(0 \X(t) - X(t) + E(0 V(0 . 



(5.4) 



On the other hand, the equation giving the optimal estimate is, according to (4.86): 



x(0 = A(0x(0 + B(0u(0 + K(0[y(0 - C(0x(0j 

= A(t)x(t) + B(f)u(0 + K(t)C(t)\x(t) -x(0l + K(?)W(0 . 



By subtracting this equation from the first of (5.3), we get 



X(0 = | X(0 - x(t) | = A(t) | X(0 - x(t) | + E(t)\(t) - K(t)C(t) | X(t) -x(OJ 

-K(?)W(0, 



i.e., 



X(0 = [A(0 - K(0C(0] X(0 + E(?)V(0 - K(?)W(0 . 
Regroup now equations (5.4) and (5.5): 
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This writing shows clearly that the dynamics of the estimation error is completely 
decoupled from that of the state. The stochastic separation theorem takes thus here 
exactly the same form as the algebraic separation theorem, encountered for the de- 
sign of a state-feedback control system containing an observer in the loop to re- 
construct the state. 

The functional diagram of such a control system in represented in Fig. 5.1. 



White noise W(t) 




Deterministic optimal controller 
Fig. 5.1 Functional diagram of a state-feedback control system including a Kalman-Bucy filter. 



5.4 Loss of Robustness in an LQG Synthesis 



The inclusion of an estimator in an LQC-type control loop can lead to a loss of ro- 
bustness. Since this goes beyond the scope of this book, the interested reader may 
consult the specialized works of this field, such as [Doy78] and [M0GB8I]. 

An illustration of this phenomenon is proposed in an exercise at the end of this 
chapter. 
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5.5 Solved Exercise 



Exercise 5.1 Discrete-time LQG Control of an Air Flow 
Heater 

We consider again the air-flow heater used in Exercises 3.6 and 4. 1. We want here 
to apply an LQG control to this setup. 

a) Design an optimal LQC control for the discrete model of this setup, with 
the following weighting matrices: Q = I 2 ; R = 10 . 

b) Calculate a Kalman filter with the following covariance matrices of the 
process noise and the measurement noise: Q = 100 ; R = 1 . 

c) Simulate the closed loop with Simulink®, and verify its operation with the 
Kalman filter included in the loop. 

d) Study the robustness of the obtained LQG control, by following the same 
approach as in question (c) of Exercise 3.6. What do you remark? 

Solution: 

(a) The program MMCE.m yields readily the following control law: 

£ J = (-0,2729 0,1427) ; M = 1,8689 . 

(b) The same Kalman filter as in Exercise 4.1, question (a) is found. 

(c) The responses are quite similar to those of Exercise 4.1, question (a). 

(d) The script file LQ_LQG_robustness_discrete.m, which is included in 
the archive mmce.zip, plots in its second half, entitled 2) Study of the LQG 
control robustness (filter in the loop), the compensated open-loop Nyquist dia- 
gram and the "robustness" circle. 

The statement LQGREG(KEST,L, 'current') produces an LQG controller by 
connecting in series the Kalman filter KEST calculated by kalman. m and the state 
feedback matrix L calculated by dlqr, as sketched in Fig. 5.2. 
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KEST 



Fig. 5.2 Block diagram for the MATLAB® statements kalman.m and dlqr. 

This controller is then connected in series with the plant output to build the com- 
pensated open loop, by the statement Topen_ss = series(PT326_ext, RLQG). 
Finally, the Nyquist diagram of the compensated open loop is calculated by the 
statement [Re,lm] = nyquist(Topen_ss, omega), and the forbidden circle is cal- 
culated according to (3.92). The obtained plot is represented in Fig. 5.3. 




Fig. 5.3 Compensated open-loop Nyquist diagram (dashed line), forbidden disk 
(hatched) and unit circle (dash-dotted line). 



At the difference with the plot obtained at question (c) of Exercise 3.6, it appears 
here that the Nyquist diagram of the compensated open loop enters the forbidden 
disk. The guaranteed robust stability margins are therefore lost, as a consequence 
of the introduction of an estimator in an LQC control loop. 



6 Linear Matrix Inequalities 



The origin of Linear Matrix Inequalities (LMIs) goes back as far as 1890, although 
they were not called this way at that time, when Lyapunov showed that the stabil- 
ity of a linear system x = Ax is equivalent to the existence of a positive definite 

matrix P, which satisfies the matrix inequality A T P + PA < , expression which 
will be clarified below. The term "Linear Matrix Inequality" was coined by Wil- 
lems in the 1970's to refer to this specific LMI, in connection with quadratic op- 
timal control. Due to the lack of good computers as well as of efficient algorithms 
to solve them, the LMIs did not receive a great deal of consideration from control 
and system researchers until the late 1980's, when Nesterov and Nemirovsky de- 
veloped interior-point methods that allowed solving elegantly LMI problems. New 
algorithms appeared then, triggering a renewed interest in this subject. 

The purpose of this chapter is to give the basics of these methods and their 
utilization in control theory, with applications in the solved exercises at the end of 
the chapter. The reader can perform the solutions with the downloadable software, 
as is the case for all the exercises of this book. 

Details about LMIs and their history can be found in books or publications, 
such as [BBFE93], [BEFB94], [ANAL06], and [Duc02b]. 



6.1 General Considerations and Main Properties 
6.1.1 Definition of an LMI 

A linear matrix inequality is a constraint of the form: 

m 

F(x)^F +J2^,>0, (6.1) 

where 
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x m ) e : 



6 Linear Matrix Inequalities 
is the vector of the m variables, 



• F ; = F ; 6K" X " are given symmetric matrices, 

• the inequality symbol, > , means that the matrix F(x) is positive definite, i.e., 
u T F(x)u > for all nonzero u€l" (see Appendix B, Sect. B.6). 

There are also nonstrict LMIs, of the form F(x) > , where > means that the 
matrix F(x) is positive semidefinite, and LMIs of the form F(x) < which are 
obviously equivalent to — F(x) > . 

A set of constraints, or multiple LMIs, such as F^(x) >0,...,F (p \x) >0, can 
be grouped into a single LMI: 










F 2 (x) 













>0. 



In the sequel, there will be no distinction between a set of LMIs and a single LMI. 

Remark 6.1. When the matrices F, are diagonal, the LMI F(x) > becomes sim- 
ply a set of affine inequalities. 



6.1.2 Basic Properties of LMIs 



Property 6.1. Given a nonsingular matrix T = T' e K" x " , the LMI F(x) > is 
equivalent to T T F(x)T > . 

Proof. The proof is trivial, if one considers that the quadratic form u T T T F(x)Tu 
is the same as v T F(x)v , with v = Tu . Therefore, if the matrix F(x) is positive 
definite, T T F(x)T is it also, and vice-versa. 

Property 6.2: Schur Lemma 

Given the following matrices: Q(x) = Q(x) T e K' ,x " 

and S(x) eR"", which depend affmely on x, the LMI 

'Q(x) S(xY 

S(x) T R(xX 



>0 



R(x) = R(x) T eR mxm , 



(6.2) 



is equivalent to the two LMIs 
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(R(x)>0 
1q(x)-S(x)R(x)- 1 S(x) t >0 
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(6.3) 



and to the two LMIs 



Q(x)>0 
R(x)-S(x) T Q(x)- 1 S(x)>0 



(6.4) 



In other words, the set of nonlinear inequalities (6.3) or (6.4) can be represented 
by the LMI (6.2) 

Proof. Let us prove the first equivalence, the second one being contained in the 
first one by simple substitution of matrices. The condition R(x) > is contained 

implicitly in (6.2), since R(x) is one of the diagonal blocks of a positive definite 
matrix. With the use of Property 6.1, and by dropping temporarily the dependen- 
cies on x to simplify the notations, we can write: 



X 


' 


Q 


s N 


X 


o N 


t 





R 1 


s T 


R 





R ~\ 





Q 

R"'S T 



SR" 
R" 1 



>0, 



where I„ is the identity matrix of size (nxn) . By applying again Property 6.1 to 
the right side of this matrix inequality, the resulting inequality 



X 


-s' 


Q 


SR" 1 


X 


o x 




{ 

Q 


-SR _1 S T 


o' 





R 


RV 


R" 1 


-S T 


R 









R 



>0 



is effectively the same as (6.3). 

As illustrated above, this result is very useful to convert nonlinear matrix ine- 
qualities to LMI form. An interesting example is given by the following con- 
straint: 



tr[s(x) T P(x)- 1 S(x)]<l, P(x)>0, 



where P(x) = P(x) T e M. nx " and S(x)er xp depend affmely on x. This con- 
straint is treated by introducing a new matrix variable Z = Z T G K pxp , called a 
slack variable. The previous constraint is then equivalent to the LMI in x and Z: 



tr(Z)<l, 



T\ 



Z S(x) 

lS(x) P(x)j 



>0. 
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Matrices as variables. Many problems will be encountered in which the vari- 
ables are matrices, e.g. the Lyapunov inequality already mentioned at the begin- 
ning of this chapter, 



A'P + PA<0, 



(6.5) 



associated with the stability of a linear system, having the system matrix 
A € W x " in state representation. Here the variable, also called decision variable 
in the LMI terminology, is the symmetric matrix P = P T . Even though this LMI 
could be written explicitly in the from (6.1) by a suitable choice of F and the F ; , 

it is more interesting to keep the condensed form (6.5), and to indicate which ma- 
trices are the variables in a phrase, e.g. in this case, "the LMI (6.5) in P". 

Another typical example is the following quadratic matrix inequality, or Ric- 
cati matrix inequality, which will be encountered later again: 



A T P + PA + PBR ~B T P 



Q<o, 



(6.6) 



where A, B, Q = Q T , R = R T > are given matrices of appropriate sizes, and 

P = P T is the variable. By using the Schur Lemma, this quadratic matrix inequal- 
ity in P can be transformed into an LMI in P, since it can be rewritten as the fol- 
lowing set of two constraints: 



-R<0 

A T P + PA + Q (-PBR _1 B T P) < 



i.e., with the equivalence between (6.3) and (6.2), 



A'P + PA + Q PB 
B T P -R 



<0. 



Property 6.3: Elimination Lemma 

Consider the following LMI in X, of some particular form: 



Q + UXV T +VX T U T <0 



(6.7) 



with Q = Q T GK" X ", UGK" xm , rank(U)<«, V eR nxp , rank(V) < n , and 
XeR mxp . 

We intend to eliminate X from this LMI. Assume that U ± is an orthogonal 
complement of U, i.e. a matrix of size (n — m)xn such that (see Sect. B.4, in Ap- 
pendix B) 
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U ± U = and rankfu^) 1 ] = n , (6.8) 

and that similarly V is an orthogonal complement of V, i.e. a matrix of size 
(n — p) x n such that 



V x V = and rank V(V ± ) T \ = n . (6.9) 



The LMI (6.7) has then a solution if and only if: 

[U- L Q(U ± ) T <0 

V ' (6.10) 

[V ± Q(V ± ) T <0 

Proof. The proof of the only if 'part of this lemma is trivial, by left and right mul- 
tiplying (6.7) successively by U ± and (U ± ) T , and then by V ± and (V x ) .The 
proof of the if part is more involved and can be found in [BEFB94]. 

Remark 6.2. We have limited ourselves here to the minimum properties, which 
will be useful for the sequel of the material presented in this chapter. There are 
many more properties of LMIs, which can be found e.g. in [BEFB94], [ScWe06]. 



6.2 Standard Problems Involving LMIs 

There are two main classes of optimization problems with constraints, expressed 
as LMIs. 



6.2.1 LMI Feasibility Problems 

This problem consists in finding an x € M m solution to the LMI F(x) > , or to 
determine that this LMI is infeasible, i.e. that no such x exists. 



6.2.2 Eigenvalue Problems 

The eigenvalue problem consists in minimizing the eigenvalue of a matrix, which 
depends affinely on some variable, subject to an LMI constraint: 
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minimize A 

subject to AI - A(x) > 0, B(x) > 

Such problems appear often in the equivalent form of minimizing a linear func- 
tion of x subject to an LMI: 

minimize c T x 
subject to F(x) > 



6.3 LMI Problems in Systems and Control 



6.3.1 Stability of an Homogeneous Linear System 
(Lyapunov) 

Given an homogeneous linear system with state vector x, the theory of Lyapunov 
claims that, if there exists a scalar function V(x) verifying simultaneously 

(Or(o)=o, 

(//) K(x)>0,Vx^0, 

(Hi) V(x) is decreasing, for any x * , 

then the equilibrium point x = is asymptotically stable, i.e. the response to an 
arbitrary initial condition x(0) tends asymptotically towards zero. 

A reasonable choice for the Lyapunov function is the quadratic function 
V(x) = x T Px , with P G K" x ", P = P T > . The two first conditions are fulfilled 
by this function, obviously. From the positive definiteness of P it results that 
Vx T Px = sJV(x) is a weighted norm, which can measure the distance to the ori- 
gin x = [ANAL06]. It is therefore quite natural to consider that the system is 
stable if the distance to the origin is always decreasing, which represents the third 
condition. 



6.3.1.1 Continuous Case 

Since in this case an homogeneous system is described by x = Ax , x G W , the 
third condition expresses as 
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V(x) = x T P x + x T Px = x T A T P x + x T PA x 

= x T (A T P + PA)x<0, Vx^O 

Therefore, the equilibrium point x = of this system is asymptotically stable if 
there exists a matrix P = P T 6R° X ° satisfying simultaneously the two LMIs 



P>0 

A T P + PA<0 



(6.11) 



Remark 6.3. Though not fundamental, it is interesting to recognize that this is in- 
deed an LMI, as defined in (6.1). By defining P as the matrix whose elements 

are all zero except elements (i, j) and (j, i) which are equal to one, we can write 

n i 

P = y^y^ Q^-P,-,- , and the two previous inequalities become: 

i=l ;=1 



n i 
i=l 7=1 



Py 

-A T P y -P y A 



>0, 



which is indeed an LMI with a vector x including all the coefficients cu, . The 

writing (6.11) in terms of an LMI in the matrix variable P is of course much more 
concise, and justifies the use of matrix variables as introduced in Sect. 6.1.2. 



6.3.1.2 Discrete Case 

An homogeneous linear system being described here by x t+1 = Ox t , x G R" , the 
third Lyapunov condition writes 

V(x k+l ) - V(x k ) = x T k+l P x k+l - x T k P x, = x T k <D T P O x, x T k P x, 
= x J k (0 J VO-V)x k <0, Vx^^O 

Therefore, the equilibrium point x = of this system is asymptotically stable if 
there exists a matrix P = P T 6R" X " satisfying simultaneously the two LMIs 

P>0 

T • (6-12) 

O T PO-P<0 
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6.3.2 Stabilizability and Stabilization by State Feedback 

6.3.2.1 Continuous Case 

Consider the system defined by x = Ax + Bu . Does there exist for this system a 
state feedback u = — Lx (see (1.2)) such that the closed loop system is stable? 

To answer this question, let us apply the above Lyapunov theory. Since accord- 
ing to (1.3) the homogeneous closed-loop system obeys the state equation 
x = (A BL)x, this system is stable, according to (6.11), if there exists a sym- 
metric matrix P £ R nxn such that: 



P>0 

(A BL) T P + P(A BL)<0 



(6.13) 



where the decision variables are P and L. Note that the second matrix inequality is 
not linear, thus not an LMI, since it contains products of these two matrices. 

We will now apply a procedure, which will be used many times in the sequel. 
First, noting that the constraint P > guarantees that P~ exists and is nonsingu- 
lar, multiply left and right the two inequalities (6.13), according to Property 6.1, 
by P" 1 and its transpose, (P -1 ) T =P -1 : 

P-'PP-'X), /.e.iP-'X) 

P- 1 (A-BL) T +(A-BL)P -1 <0, 

/e:P "A T +AP _1 P "L T B T BLP" 1 <0 

With the simple change of variables Y = P~ and W = LP" = LY , the follow- 
ing LMIs in Y and W are now obtained, after some rearrangement: 

Y>0 

T T T (6-14) 

AY + YA T -BW-W T B T <0 

If these LMIs have a solution (i.e. are feasible), a state feedback gain which stabi- 
lizes the system is given by L = W Y _1 . 

a-Stabilization by state feedback. If in addition to the stabilization requirement 
it is desired that the closed-loop eigenvalues have all a real part inferior to 
—a < , the closed-loop transient response will decay faster than e~ at . The 
closed loop is then said to be a-stable, or also to have exponential stability. 
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It suffices to note that the a-stability of the closed-loop system x = (A — BL)x 
is equivalent to the stability of the closed-loop system x = (A' — BL)x , in which 
the matrix A' = A + a\ n is obtained from A by shifting the real part of its eigen- 
values. Indeed, since det(sI B — A') = det [{s — a)\ n — A] = det(<rl n — A) , where 
a = s — a , the eigenvalues of A — BL will all have a real part inferior to —a if 
all those of A' — BL have a negative real part. 

From (6.14) it follows then that a matrix L solution of the present problem is 
L = WY" 1 , with Y and W solution of the LMI 

Y>0 

T T ( 6 - 15 ) 

( A + a\ n )Y + Y(A + c& H ) T - BW - W T B T < 

Decay rate of a continuous-time system. The decay rate of a system x = A x is 
the largest value of a > for which this system is a-stable. In other words, —a 

is the real part of its rightmost eigenvalue. To find this value, it would be enough 
to solve the following optimization problem: 

minimize — a 

subject to (6.16) 

A T P + PA< 2aP, P>0, a>0 

Unfortunately, because of the product aP, this is not an LMI, and (6.16) can be 
solved only by dichotomy methods or with bilinear matrix inequality solvers, 
which goes beyond the scope of this chapter. 



6.3.2.2 Discrete Case 

Applying a state feedback control u t = — L x k to a discrete-time linear system 
x t+1 =Ox t +ru t yields the following homogeneous closed-loop system: 
x t+1 =(0-rL)x t (see (1.13)). According to (6.12), this system is stable if there 
exists a symmetric matrix PeE" x ° such that: 

P>0 

(<D TL) T P(0 TL) P<0 

Using the Schur Lemma of (6.3) and (6.2), these two matrix inequalities are 
equivalent to the following single matrix inequality: 
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(<D TL) 



T\ 



O-TL 



<0. 
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(6.17) 



This inequality is not an LMI, because of the simultaneous presence of terms in P 
and P" 1 . By using again Property 6.1, but pre-multiplying and post-multiplying 



now this inequality by the symmetric, nonsingular matrix 



p _1 (o-rL) TN| 



(o -rL)p _1 



<0. 



we obtain: 



With the same change of variables as above, namely Y = P ' and W = LY , the 
following LMI in Y and W results finally: 



-Y Y<D 

OY-TW Y 



W T r T ^ 



<0 



(6.18) 



If this LMI has a solution, i.e. is feasible, the state-feedback gain matrix 
L = WY" 1 stabilizes the closed-loop system. 

a-Stabilization by state feedback. A discrete-time system is said to be a-stable, 

or also to have exponential stability, if all its eigenvalues are contained within a 
disk of radius l/a , with a > 1 . 

We note that the a-stability of the closed-loop system \ k+l =(0-TL)x t is 

equivalent to the stability of the closed-loop system x t+1 =(0' — TL)x /t , in 

which the matrix O' — aO is obtained from O by multiplying its eigenvalues by 



a . Indeed, since det(zl n — <&') = det 



I O 



I ot 



■ a" det(£I„ - O) , where 



£ = z/a , the eigenvalues of O — T L will all have a modulus inferior to l/a if all 
those of O' — r L are within the unit disk centered at the origin. 

From (6.18) it follows then that a matrix L solution of the present problem is 
L = WY'', with Y and W solution of the LMI 



aYO 1 



\v T r T] 



aOY TW 



<0. 



(6.19) 
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Decay rate of a discrete-time system. The decay rate of a system x i+1 = Ox t is 

the largest value of a > 1 for which this system is a-stable. In other words, l/a is 
the modulus of its eigenvalue closest to the unit circle. 



6.3.3 Detectability and Detection of Linear Systems by 
State Observer 

The notion of detectability is dual to that of stabilizability (see Appendix A, 
Sect. A.3.3), exactly as observability is dual to controllability, as was shown in 
Sect. 2.2.2 and 4.6. 

We claim that the following results are true. 

Continuous case. If the following LMIs in Y and V, 

[Y>0 

T T (6.20) 

A T Y + YA-VC-C T V T <0 

are feasible, the observer with system matrix A — GC, where G = Y _1 V "de- 
tects" the plant state. 

Discrete case. If the following LMI in Y and V, 



-Y YO-VC) 

O t Y-C t V t Y 



<0 , (6.21) 



is feasible, the observer with system matrix O — GT, where G = Y ! V, "de- 
tects" the plant state. 

Proof. These LMIs are readily obtained by replacing (A — BL) by (A — C T G T ) 
in (6.13) for the continuous case, or (O-TL) by (O t -C T G T ) in (6.17) for the 
discrete case, and following then the same procedure as in Sect. 6.3.2 with the 
change of variables Y = P and V = YG . It may be interesting to note that the 
duality can be extended, in saying that to W of the stabilizability problem corre- 
sponds V T = G T Y of the detectability problem. This would have allowed to write 
(6.20) and (6.21) directly by duality from (6.14) and (6.18). 
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6.3.4 LQC Problem Solved with LMIs 
6.3.4.1 Continuous Case 

The linear quadratic control problem, which has been discussed in Sect. 3.5.6, is 
the following. Given a linear system, defined by 



i(0 = Ax(0 + Bu(0, AeK" x ", Be: 



5 n x P 



with x(t ) = x specified, find a state feedback u(t) = — Lx(7) which minimizes 
the following quadratic criterion: 



J = 



f [x T (t) Q x(0 + u T (t) R u(0] dt , 
Jo [ ' 



where Q = HH>0 and R = R 1 >0, with HeR' x ", ? = rank(Q), and 

R e K ; ' x; ' , as seen in Sect. 3.5.6, apart from the factor 1/2 which is unimportant 
as already mentioned in that section. 

We saw in that section that the solution is obtained from the unique positive 
semidefinite solution of the algebraic Riccati equation (3.54). 

In the following, we will be looking for a state feedback u(t) = — L \(t) which 

guarantees that the criterion J is inferior to some given number 7 [Duc02b]. 

Since u T Ru = x T L T RLx, let us introduce the function F(x) = x T Px, with 
P = P T > , satisfying the two following conditions: 

• ' T T (6.22) 

V(x) + x T Q x + x T L T R L x < 

Such a function V(x) is a Lyapunov function, since it satisfies all three conditions 
of Sect. 6.3.1. Furthermore, 

/ V(x)dt+ \ (x T Qx + x T L T RLx)<A<0, (6.23) 

Jo Jo 

which we can rewrite as 

I (x T Qx + x T L T RLx)^<F(x ) = xJPx < 7 . (6.24) 

Jo 

J 
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If there exist a matrix L and a function V(x) satisfying (6.22), then L solves this 
problem. By recalling that V(\) = x T Px + x T Px and taking into account the 
closed-loop state equation, these two inequalities are equivalent to 



Jx Px < 7 

l(A BL) T P + P(A BL) + Q + L T RL<0 



(6.25) 



By left and right multiplying the second inequality by Y — P and introducing 
W = LY , we obtain successively the following inequalities: 



Y(A BL) 1 +(A BL)Y + YQY + YL'RLY<0. 



YA T + AY BW W T B T + YH T HY + W T RW < , 



YA T +AY-BW-W T B T + YH T W T 



R 



HY 

W 



<0. 



By applying the Schur Lemma, this inequality becomes the following LMI: 



YA T +AY-BW-W T B T YH T W T ^ 



HY 

W 



R 



<0 



(6.26) 



The first of the inequalities (6.25) becomes, successively: 



7 -x Px >0, 



7-xjY- 1 x >0, 



7 x 
x o 





Y 



>0 



(6.27) 



with the use of the Schur Lemma again. Note that the initial constraint P > , i.e. 
Y > , is contained in (6.27). A state feedback matrix L, solution of the problem, 

is thus obtained by solving the LMIs (6.26) and (6.27), and letting L = WY" 1 . 

In order to get rid of the knowledge of x , the condition (6.27) can be replaced 

by the condition P — jl n < . This guarantees that, for any x , 

/ < xJPx < 7xjx . The previous condition becomes then jl n — Y _1 > , i.e., 



280 



7l I ) 

In n 

i„ y 



>o 
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(6.28) 



If it is desired to minimize the value of 7 , the following optimization problem, 
which is a typical eigenvalue problem as defined in Sect. 6.2.2, should be solved: 



mm 7 

7 ,Y=Y T ,W 

subject to (6.26), (6.28). 



(6.29) 



6.3.4.2 Discrete Case 

Given a discrete -time system described by 

i i+1 = «i t +ru t , oeK" x ", reR" xp , 

and a quadratic criterion 

^ = E( x *Q x *+ u I Ru *)' 

4=0 

the goal is to minimize J by means of a state feedback u k = — L x t , all other hy- 
potheses being the same as in the continuous case. Let us choose here a function 
^(x^) = xJP Xj. satisfying the two following conditions: 

JF(x )< 7 

[V(x k+l )-V(x k ) + x J k Qx k +x J k l7RLx k <0 

The inequalities (6.23) and (6.24) are replaced by 

00 00 

= k = 

OO 

£>JQx 4 +xJl t RLu 4 ) < F(x ) = xJPx <7, 



i = 



and (6.25) becomes here 
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JxJPx < 7 

|(* TL) T P(* TL) P + Q + L T RL<0 
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(6.30) 



Multiplying again left and right the second inequality by Y = P l and introduc- 
ing again W = LY , this inequality writes 



(Y* T W T r T )Y _1 (*Y - T W) - Y + YH T HY + W T R W < , 



Y (Y* T W T r T YH T W T 



-Y 







-I 
-R 



(#Y-rw) 

HY 

W 



<o, 



and, with the Schur Lemma, 



Y 


Y* T 


-w T r T 


YH T 


w 


*Y-rw 




Y 










HY 









-h 





w 












R 



<0 



(6.31) 



The first inequality of (6.30) being the same as in the continuous case, (6.28) 
holds here also. 

If it is desired to minimize 7 , the following optimization problem should be 
solved: 



mm 7 

7,Y=Y T ,W 



subject to (6.31) and (6.28) 



6.3.5 Pole Placement in LMI Regions 



Pole assignment in convex regions in the left-half plane or in the unit disk can be 
expressed as LMI constraints, either applied to the Lyapunov matrix P involved in 
such a single constraint, or added to other constraints, e.g. in the case of an LQC 
synthesis with regional pole constraint. 

We will follow here the basic developments of [ChGa96], but with the nota- 
tions used in [ScGC97]. Early work in this field has been done by [ArBC93]. 
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Kronecker product of matrices. The Kronecker product of two matrices 
P = (p u ) e M. mx " and Q = (q tj ) e R pxq , denoted by , is defined as follows: 



P0Q : 



JmlQ ■■■ PmnQj 



LMI region. An LMI region is any region TZ of the complex plane that can be de- 
fined as 



K ={ 



= z€ ( 



■zM, 



-zM 



T | 
reg J 



<0. 



(6.32) 



where L, = L. and M,„„ are fixed real matrices [ScGC971. Note that 1Z is a 

convex region in the complex plane, symmetric about the horizontal axis. Such a 
region can be a vertical strip, a disk, an horizontal strip, a conic sector, an ellip- 
soid, a parabola, and an arbitrary intersection of such regions. Details can be found 
in [ChGa96]. Let us give just a simple example. 

Example: disk of radius r and center (— q, 0) . A point in the complex plane repre- 
senting the complex number z belongs to this disk if and only if \z + q\ < r , i.e. 

I l2 -> -> 

\z + q\ <r , which we can also write — r + (q + z)(q + z ) < , or equivalently, 

r being strictly positive, — r + (q + z) r~ l (q + z ) < . With the Schur Lemma, this 
inequality can be cast in an LMI: 



-r q + z 
T z~ —r 



<0. 



By comparison with (6.32), this region is characterized by the following matrices: 



-r q 



M. 



1) 




The matrices characterizing many of the above listed LMI regions are con- 
tained in the script file yalmip_lmi_reg.m included in the downloadable software 
accompanying this book. Note that the matrices characterizing a convex region de- 
fined by the intersection of several LMI regions are obtained by aligning on the 
diagonal the L reg and M reg matrices corresponding to the individual LMI re- 
gions, as results from the grouping property of a set of LMIs into one single LMI 
(Sect. 6.1.1). 
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The Lyapunov theory, which applies for the open left-half plane 
{z : z + z < 0} can be generalized to any LMI region. The eigenvalues of a given 

matrix A are all contained in the LMI region 1Z if and only if some LMI involving 
A is feasible, as specified by the following theorem [ChGa96]. 

Theorem 6.1. The matrix A has all its eigenvalues in the LMI region (6.32) if and 
only if there exists a symmetric matrix P such that 

L reg (g)P + M reg (8)A T P + M^(8)PA<0, P>0 . (6.33) 



6.3.6 State Feedback with Regional Pole Placement 

The system matrix concerned with the assignment of poles in LMI regions is here 
the closed-loop matrix. In case of state feedback, this matrix is A — BL (continu- 
ous case) or O — TL (discrete case). If this matrix is substituted in (6.33), this 

inequality in the decision variables L and P is no longer an LMI, because of the 
product LP and its transpose. Using the same procedure as in Sect. 6.3.2.1, we 
multiply this inequality left and right by P~ and introduce again Y = P and 
W = LY . The following LMI is obtained in the continuous case: 



L reg ® Y + M reg (AY - BW) ' + M, 1 ^ ® (AY - BW) < 0, Y > . (6.34) 
The LMI of the discrete case is the same, with A and B replaced by O and T . 

6.3. 7 Inverse Problem of Optimal Control 

The inverse problem of optimal control is the following. Given a matrix L, deter- 
mine whether there exist two matrices, Q > and R > , such that (Q, A) in the 

continuous case, respectively (Q,0) in the discrete case, is detectable and 
u = — Lx is the optimal control for the corresponding LQC problem. 

6.3.7.1 Continuous Case 

Given a continuous-time system x = Ax + Bu, x(0) = x , and a matrix L. We 
are seeking Q > and R > such that there exist P > and Pj > satisfying 
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(A BL) T P + P(A BL) + L T RL + Q = 

B T P RL = 

A T P 1 +P 1 A<Q 



(6.35) 



Since L is given, the constraints involved here are two linear matrix equalities and 
one LMI, the decision variables being P, P b Q and R. The first constraint repre- 
sents the continuous-time ARE (3.54), rewritten for linearization purposes as 

PA PBR ! B T P + A T P PBR _1 B T P + PBR >B T P + Q = 
P(A -BL) + (A BL) T P + L T RL + Q = 0, 

with L = R 'B T P as given by (3.55), which is reflected in the second constraint. 
The LMI is equivalent to the condition of (Q,A) being detectable [BEFB94]. If 

the L matrix used is the one obtained by solving (3.54) and (3.55) with some arbi- 
trary pair of weighting matrices, the matrices Q ipoc and R ipoc solution of (6.35) 
should yield the same value of L if the ARE problem is now solved with them. 



6.3.7.2 Discrete Case 

Assume given a discrete-time system \ k+1 = 0\ k + Tu k , x(0) = x , and a ma- 
trix L. By transposition of the previous results to the discrete situation, we seek 
Q > and R > such that there exist P > and P[ > satisfying 



(O TL) T P(0-rL) P + L T RL 

r T PO (R+r T pr)L = o 



-Q = 



(6.36) 



T P 1 0-P 1 <Q 



The decision variables are P, P 1; Q and R. The first constraint represents the dis- 
crete ARE, with L = (R + r T Pr)"'r T PO (see (3.84)) reflected in the second 
equality. The LMI is equivalent to (Q,3>) being detectable. 



6.3.8 Extension to Uncertain Plants. Robustness 



A very interesting feature of LMI methods is that they can convert elegantly a 
nominal design problem into a robust design. Assume, for example, that for some 
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real plant several linear models have been developed, corresponding to different 
operating points, maybe also from experimental data taken at different times: 

x. = Ax + Bu, i = \,...,n M . 

Suppose that the following robust control design is submitted: determine a ma- 
trix L such that the state feedback u = Lx stabilizes all these n M models. This 
amounts to imposing that the closed-loop systems x ■ = (A . — B, L)x, i = l,...,n M 

are stable. From (6.14) we can conclude that, if there exist Y = Y T >0 and 
W = W > such that the following LMIs are feasible 



Y>0 

A, Y + YA^ B, W W T B* < 0, i = 1, 



(6.37) 



then the matrix L = W Y solves the problem. 

An identical synthesis is of course also possible for discrete-time uncertain 
plants, by extending this time (6.18) to n M models of the plant. 

It may be worthwhile to note that, to obtain in the above example one single 
controller it was mandatory to have only one Lyapunov matrix Y and one matrix 
W in the LMI (6.37), the same for all n M partial LMIs. This represents some re- 
striction to the use of LMIs for robust synthesis problems. Note also that the above 
approach, which worked well for a state-feedback synthesis, could not be used e.g. 
to treat an output-feedback design problem. 



6.4 Software Tools to Solve LMIs 

To solve under MATLAB® the LMIs presented in this chapter and used in the fol- 
lowing exercises, a solver is needed. A powerful LMI solver is the freeware Se- 
DuMi [Stu99]. Several versions are downloadable from the Internet sites indicated 
in the reference. According to their authors, Version 1.21 has been tested and con- 
firmed to run under MATLAB® versions up to 2009a. The newly released version 
1.3 supports MATLAB® versions from 2007b to 2009b. From version 1.21 on, 
SeDuMi runs also on 64-bit operating systems. Installation instructions are in- 
cluded in the downloadable archive. The exercises proposed hereafter can also be 
solved with CVX, an alternative freeware package for specifying and solving con- 
vex problems [GrBo08], [GrBolO], which includes SeDuMi. 

A very useful free interface with MATLAB® is provided by Yalmip [L6f04]. 
The download link is given also in the reference. The user is invited to read care- 
fully the installation instructions on this site. Yalmip relies on many external 
solvers, among which SeDuMi. 
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6.5 Solved Exercises 

The first exercise is based on the same plant as used in previous chapters, to give 
the reader the possibility to compare with previous results, where comparisons are 
justified. The second one is more specific to the matter of the present chapter. 



Exercise 6.1 Inverted Pendulum 

Consider again the inverted pendulum of Exercises 1.1, 2.1 and 3.4. By using ex- 
clusively LMI methods, except in question (e), solve the following questions. 

a) Determine a continuous-time state-feedback control that guarantees an te- 
stability for the closed loop with a — 5 . Compare with the results of Exer- 
cise 1.1, question (b). 

b) Determine an LQ controller for the continuous-time model with the state 
weighting matrices of the first choice of Exercise 3.4, question (a), i.e., 
Q = 100xl 4 , R = \. Determine the rise time of the closed loop step re- 
sponse. Comment. 

c) Calculate a state-feedback control with regional pole placement, the con- 
finement region being defined by the intersection of the half plane to the 
left of x Q = —5 and a disk of radius 10 centered at 0. 

d) Design an LQ controller with regional pole placement, with the same 
weighting matrices as in question (b) and the same pole confinement speci- 
fication as in question (c). Check in simulation the closed-loop step re- 
sponse and the corresponding control signal. Compare with the result of 
question (b) and comment. 

e) Design an LQ controller L ( by quadratic criterion (solution of the alge- 
braic Riccati equation) with the same weightings as in question (b). Deter- 
mine then a pair of weighting matrices Q^ oc and R,^ oc solution of the in- 
verse problem of optimal control, i.e. which yield this optimal control 
L t . To verify that, make without exiting the program a new LQC design 

by solving the ARE with weightings Q = Q ipoc and R = R^ oc . Compare 
this new L matrix with the L ( calculated at the beginning of the question. 

f) Repeat question (e) with the discrete -time model of the pendulum. 
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Solution 

(a) Run MMCE.m with the inverted pendulum, two measurements and a 
continuous model. Then select LMI methods, unique component of y to be 
set to its reference value: 1, no integral action, alpha stabilization by state 
feedback, and value Of alpha: 5. The state feedback controller and gain compen- 
sation matrices, obtained by solving (6.15), are: 

£ J = (-69.84 18.91 48.19 3.894), M = -69.84, 

and the closed-loop poles are: —6.204 ±j 2.210, — 11. 18±y 15.96 . They have 

all a real part less than —5, as required. In comparison with the results of Exercise 
1.1, a step response at least as fast is obtained in simulation, with some overshoot, 
which was not the case in Exercise 1.1. This is due to the existence of complex 
poles here, while only real poles were obtained in Exercise 1.1. Moreover, the sig- 
nificantly more negative real part of the second complex pole pair, in connection 
with significantly larger coefficients of the feedback gain matrix, result in a much 
stronger control signal (—70 V initial value!). The present synthesis does not im- 
pose, indeed, any limit to the location of the poles towards high negative real parts 
or to high moduli. 

(b) Without leaving the program, go through the same steps as in question (a) 
until the LMI methods menu is reached, and choose now LQ regulator design. 
Enter the specified weighting matrices. Solving the LMIs (6.26) and (6.28) yields 
the LQ controller and gain compensation matrix 

£ T =(-10.01 34.32 32.41 12.41) ; M = -10.01 . 

Within an accuracy of about 0.1% the results are identical to those of Exercise 
3.4, question (a), case (1). The dominant closed-loop pole is s = — 1.38, which 
accounts for a rather slow rise time (time of first maximum: 4 s). 

(c) Without leaving the program, go through the same steps as in the previous 
questions until the LMI methods menu is reached again, and choose now State- 
feedback design with regional pole placement, and then, for the LMI region, 
select successively h) Half-plane, I (left), xO = -5, d) Disk, Abscissa q of the 
center: 0, radius r: 1 0, q) Quit. The involved LMI is here (6.34) and it is solved 
by the script file yalmip_lmi_reg.m included in the package mmce.zip. The follow- 
ing matrices are obtained for the state-feedback control and the gain compensa- 
tion: 

^ T = (11.91 4.951 11.29 1.067); M = -ll. 91. 
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If we compare with question (a), the coefficients of the state feedback gain matrix 
are now much closer to those obtained by pole placement in Exercise 1.1 or by 
LQC synthesis, and the control signal stays within reasonable bounds (—12 V ini- 
tial value). The closed loop poles are now: 

-5.575 ±j 1.788; - 5.776 ±j 6.965 , 

their respective moduli obtained by abs(Poles) being: 5.854 , 5.854 , 9.048 , and 
9.048 , which indicates that the specifications are satisfied. 

(d) Still without leaving the program, go through the menus until reaching the 
LMI methods menu. Choose now LQ regulator design with regional pole 
placement, with the same choices as previously for the weighting matrices and 
the given constraints for the LMI region imposed to the closed-loop poles. By 
solving (6.26) and (6.28) again, with addition by yalmip_lmi_reg of the closed- 
loop pole confinement constraint (6.34), the program determines the following so- 
lution: 



-11.34 4.74 10.77 1.02, M 



-11.34. 



The closed-loop poles are — 5.43 1 ± 7 1.834, —5.528 ±7 7.125 , which respects 
fully the regional placement constraint. In comparison with question (b), the 
dominant pole pair (closest to the origin), has a modulus of 5.73, resulting now in 
a much faster rise time (time of first maximum: 0.7 s). 

(e) In order to dispose of an LQ controller for this question, retrieve first the 
state-feedback matrix obtained by following the steps of Exercise 3.4, question 
(a), case (1). Then, without leaving the program, proceed again to the LMI meth- 
ods menu and choose inverse problem of optimal control (ipoc). Answer yes to 
the question do you want to use the last calculated L matrix. The following 
weighting matrices are obtained: 



Q„ 



{ 0.6561 -0.1390 -0.0224 0.0206 

-0.1390 1.7460 0.4293 0.5089 

-0.0224 0.4293 1.0355 0.3910 

0.0206 0.5089 0.3910 0.7110 



R ipoc = 0.0066. 



If they are now taken as weighting matrices in a new state feedback synthesis, 
by quadratic criterion — answering this time "n" to the suggested choice of a di- 
agonal form, both for Q and R, and typing simply Q_ipoc and R_ipoc at the 
next prompts - the state feedback gain t = (-10.00 34.28 32.37 12.40) re- 
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suits, with the following closed loop poles: -845.3, -1.38, -0.92 ±j 1.09. 
These results are identical to those obtained in Exercise 3.4. 

(f) The user being now familiar with the use of MMCE.m, we leave it to him to 
treat this question in discrete -time. For the sake of verification, we give here just 
the state-feedback matrix obtained with an LQC design reusing the weighting ma- 
trices Q ipoc and R ipoc : £ J =(-0.3857 1.6253 1.5713 0.5187). 



Exercise 6.2 Uncontrollable and Unobservable Example 

The plant is here the following third order academic model, which is open loop 
unstable, not completely controllable and not completely observable: 



A = 



1 










Y 





-2 





, B = 











-4 




v i 



C = 0.5 0.5 



It is nevertheless stabilizable and detectable in this initial form. The program 
MMCE.m gives the user the possibility to change it to unstabilizable or to unde- 
tectable form, or to both simultaneously 

a) This plant being uncontrollable, check its stabilizability and determine a 
stabilizing state-feedback controller by LMI method. The plant being fed 
back directly by this state-feedback controller, observe by simulation the 
step response of the three state variables. Comment. Apply now succes- 
sively a load and a measurement disturbance, and comment your observa- 
tions. 

b) Without quitting the program, check the detectability of this plant in its ini- 
tial form and determine an observer by LMI method. The plant being now 
controlled by means of this observer and the controller determined in the 
previous question, simulate its step response again and comment. To study 
the behavior of the observer, you can let it operate in free wheel, the loop 
being closed from the state. 

c) Restart the program and change the plant model to "unstabilizable". Repeat 
the manipulations of questions (a) and (b). 

d) Restart the program and change the plant model to "undetectable". Repeat 
the manipulations of questions (a) and (b). 
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Solution: 

(a) Run MMCE.m by choosing the plant academic example, then the plant 
type 0) stabilizable and detectable, continuous model, LMI methods, with- 
out integral action, and, among LMI methods, choose stabilizability and stabi- 
lization by state feedback. The program solves then (6.14) and yields the state- 
feedback controller and gain compensation matrix 

£ T = (1.7111 -1.1829), M = 2.0136. 

The fact that the second component of t T vanishes is not surprising: x 2 is indeed 

the uncontrolled state variable as shown by the expression of B in the diagonal 
representation of the plant. The simulated step response is stable. A load distur- 
bance applied evenly to all three states does not destabilize the closed loop, as ex- 
pected since the uncontrolled mode is itself stable (A 2 =—2 ). The disturbance is 
of course not rejected, since no means have been taken for that. 

(b) Without leaving the program, go through the same steps as in question (a) 
until the LMI methods menu is reached, and choose now detectability and de- 
tection by an observer. The unobservable mode being stable (A 3 =-4), an ob- 
server is obtained by solving (6.20) with the following gain matrix: 

g T =(4.1321 -0.4147 0). 

The measurement does not influence the estimation of x 3 , as expected, this state 

variable being the unobserved one from the output as indicated by the expression 
of C m in the diagonal representation of the plant. With the observer in the loop, 

the simulated closed-loop step response is stable, as in the previous question. 
However, when the observer is operated in free wheel, it does not estimate cor- 
rectly x 3 when a load disturbance is applied, due to the lack of information given 
on that variable by g. 

(c) Though the LMI solver mentions that it detected no problem, the algorithm 
did not converge towards a usable solution, leading to the conclusion that the plant 
is no longer stabilizable, as expected, since now the uncontrollable state variable, 
x 2 corresponds to an unstable mode ( A 2 = 2 ). 

(d) The algorithm does not converge either: the plant is now undetectable, since 
the mode corresponding to x 3 is now unstable ( A 3 = 4 ). 



A State Space Representations of Systems 



The formulae and theorems recalled in this Appendix are given mostly without 
demonstration. For more details the reader is invited to consult one of the manuals 
listed in the References at the end of this book, such as [Oga02], [Ost04], 
[CAJZ01], for continuous-time systems or [AsWi97] for discrete -time systems. 



A.l State-space Representations of Linear Time- 
invariant (LTI) Systems 



A.l.l Canonical State Representations ofSISO Systems 

Among the infinite number of possible state space representations of a linear, 
time-invariant system, there are three canonical forms. They are given in Ta- 
ble A. 1 for a SISO system in connection with the parameters of its transfer func- 
tion. The continuous-time and discrete-time cases are presented in parallel in this 
table, as well as in the other tables of this Appendix. 

The two systems are assumed of order n (number of state variables required to 
describe them), which corresponds to a denominator of their transfer functions of 
degree n. For the sake of generality, the transfer functions are entered into the ta- 
ble with the same numerator and denominator degrees, which results in a coeffi- 
cient b n 3* . In most practical cases, however, due to physical reasons (finite 
bandwidth), the numerator degree is less than the denominator degree by at least 
one unit, which makes then b n = . 

We will assume furthermore that the rational fractions have been simplified by 
the coefficient a n of the denominator's highest degree term, resulting in the as- 
sumption a n = 1 . 
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Table A.l The three canonical state representations of a SISO system. 



Continuous case 



Discrete case 



Transfer function (assumption: a = 1 ): 
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Observability canonical form (assumption: a 


= 1): 










'o • 




-a 




' K \ 






'o •■■ 




-«o 




'*b ) 




x = 


i ' 




—Cl\ 


x + 


h 


u 


X A+1 ~ 


i '•. 




— «[ 


** + 


h 


" k 




o ■ 


• l 


-a„-\ , 




A-i. 






o ... 


i - 


" a »-i. 




A-i. 








A 




b 






% 




Yo 




y = 


o • 


■■ 


1 


x+ b n u 


y k = 


;o - 





1 ) 


X A + K U k 






c T 






d 


(A3) 




c e> 




d 


(A.4) 



Diagonal form (assumption: poles of G(s) or G(z) all distinct, real or complex): 
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A/.2 General Form of the State Equations (MIMO 
Systems) 



The general state representation of a system of order n, having/? inputs and q out- 
puts, is given in Table A.2. For the sake of generality, the assumption D ^ has 
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been made in this table, which is the same assumption as b n ^ in the SISO case. 

In most practical situations, however, this direct feedthrough term from input to 
output will vanish, i.e. D = . 

Table A.2 General state space representation of a MIMO system. 
Continuous case Discrete case 

i(0 = Ax(o+Bu(o (A7) Jx t+1 =*x, + rx (A8) 

y(0 = Cx(0 + Du(0 ' { y t =Cx,+Du t 

with: 

length(x) = n , length(u) = p , length(y) = q 

size(A or$) = HX», size(B or T) = n x p , size(C) = qxn , size(D) = qx p 

and: 

p<n; q <n; rank(B or T) = p ; rank(C) = q 

The sampling with zero-order hold at a period T s of a continuous-time system yields the follow- 
ing matrices replacing A and B, whereas C and D remain unchanged: 

r r s A (A.9) 

r= / e Ar rfr-B = (<D-I)A~'B, if A -1 exists 
J o 

Proof. See Sect. A.5. 



A.2 Controllability of Linear Time-invariant Systems 



A.2.1 Definition 

A linear, time-invariant system represented by (A. 7), respectively by (A. 8), where 
A, B, C and D, respectively O , T , C and D, are constant matrices, is said to be 
(completely) controllable, if its state x can be transferred, by an appropriate choice 
of the input vector u, in a finite amount of time, from an arbitrary initial state 
x(7 ) to an arbitrary final state x(^) . 



A.2.2 Controllability Criterion 



A mathematical controllability criterion is given in Table A. 3. A practical criterion 
will follow it. 
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Table A.3 Mathematical controllability criterion. 



Continuous case 



Discrete case 



The necessary and sufficient condition for the system 

x=Ax+Bu (A, B: constant) x 

to be controllable is that the (n x np) controllability matrix 



:$X(+ru t (0,T: constant) 



B AB 



(A. 10) 



is of rank n, thus has n linearly independent columns. 
N.B.: in case of asingle input system, the matrix 



b Ab 



must be regular ( <=> det Q c v± ). 



(A. 12) 



r or 



Y 3>Y 



(All) 



*" _1 y (A. 13) 



Comment about the controllability test: practical controllability. Though for 
academic examples the computation of rank(Q c ) is good enough, this may not be 
the case for real industrial plants, bearing some uncertainty in the model matrices. 
In the case Q c has a large condition number (see Sect. B.5, Appendix B), a slight 

change in one of the model parameters might change the conclusion from control- 
lable to uncontrollable. It is then safer to check this situation by applying a singu- 
lar value decomposition to Q c (see Sect. B.5), and rounding to zero too small sin- 
gular values, thus obtaining a reduced, but more realistic, rank of this matrix. An 
interesting example is given in [AlSa04], Example A.8. 



A.2.3 Stabilizability Concept 

The word completely has been put between parentheses in the previous definition 
because it is often omitted, but implicitly present notwithstanding any other indi- 
cation. Assume now that a system has a controllability matrix with a rank n c <n . 

It is thus not completely controllable. By means of an appropriate change of vari- 
able x = Tz, (see Sect. A. 7.1), it is possible to put it in the following form 

([Lar93], [ZhDG96], [DucOl]): 
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A. A 

where size(A c or O c ) = n c x n c , and where z c groups together the controllable 

state variables: the pair (A C ,B C ), or (O c ,r c ), is (completely) controllable. The 

state variables z^ are not influenced by the control u: these variables are thus not 

controllable. The above equations are the controllability canonical decomposition. 
A system is said to be stabilizable if its uncontrollable state variables have a 
stable dynamics, or, equivalently, if all its unstable modes are controllable. 

Determination of this canonical decomposition. For a system of order n for 
which rank(Q c ) = « c <«, the n c independent columns of Q c form a basis of 

the controllable subspace. The uncontrollable subspace will then be spanned by 
the n — n c vectors orthogonal to the columns of Q c , i.e. by a basis of the null 

space of Qj obtained by solving Qj x = (see Appendix B, Sect. B.4, and par- 
ticularly Property B.10). This can be done by the MATLAB® statement null(Qc') 
or, in case Q c has a large condition number, by singular value decomposition, 

svd(Qc') (see Sect. B.5). 

To establish the similitude transformation T to the above canonical representa- 
tion it is advantageous to make use of the orthogonal complementarity property of 
both subspaces (Sect. B.4), as suggested in [AlSa04]. One would then compute, 
successively, in the MATLAB® environment: 

• basis of the uncontrollable subspace: T1 = null(Qc'); 

• basis of the controllable subspace: T2 = null(T1 '). 

The transformation which yields the controllability canonical decomposition is 
then x = Tz where T = (T 2 Tj) . 



A.3 Observability of Linear Time-invariant Systems 



A.3.1 Definition 

A linear time-invariant system represented by (A. 7), respectively by (A. 8), where 
A, B, C and D, respectively O, T, C and D, are constant matrices, is said to be 
(completely) observable, if it is possible, for a given u(t) , to deduce the initial 

state x(t ) , whatever it is, from the measurement of the output y(t) during a fi- 
nite time interval t Q <t<t a . 
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A.3.2 Observability Criterion 

A mathematical observability criterion is given in Table A. 4. For practical situa- 
tions, the same comment applies as above for the controllability, by duality: the 
singular value decomposition of the observability matrix is then recommended. 

Table A.4 Mathematical observability criterion. 



Continuous case 



Discrete case 



The necessary and sufficient condition for the system 



i = AX + BU , (A, B, C,D constant) | X * 
y=Cx + Du [y k 



Cx, +Du, 



(0,T,C,D constant) 



to be observable is that the (nq x n) observability matrix 

C 

C A 

(A. 14) 

CA" \ 

is of rank n, thus has n linearly independent rows. 
N.B.: in case of asingle output system, the matrix 



c'A 



must be regular ( <=> det Q v= ). 



(A. 16) 



fc 

CO 
CO" 



V 



c t O 



(A. 15) 



(A. 17) 



A.3.3 Detectability Concept 

Consider a not completely observable system, having an observability matrix of 
rank n < n . By means of an appropriate change of variables x = Tz , it is possi- 
ble to put it in the following form ([Lar93], [ZhDG96], [DucOl]): 
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where size( A or O ) = n xn , and where z groups together the observable 

state variables: the pair (A ,C ) , or (<E> ,C ), is (completely) observable. 

The output y depends only on the state variables z , which themselves are not 
influenced by the variables z^ : these last state variables are therefore not observ- 
able. The above form is the observability canonical decomposition. 

A system is said to be detectable if its unobservable state variables have a sta- 
ble dynamics, or, equivalently, if all its unstable modes are observable. 

Determination of this canonical decomposition. The approach is the same, by 
duality, as for the controllability situation. Let us just mention here that the unob- 
servable subspace is spanned by a basis of the null space of Q , i.e. by the solu- 
tions of Q x = . Such a basis can be obtained by the MATLAB® statements 
null(Qo) or svd(Qo) (see Sect. B.5).The observable subspace, which is orthogo- 
nal to it, is spanned by the rows of Q , i.e. by the columns of Qj . 



A.4 Kalman Canonical Decomposition 



This decomposition consists in splitting the state vector into four parts ([ZhDG96], 
[AsWi97]) as sketched on the right, reflecting the four possible combinations, 
with the following subscripts: i __ l 

controllable and observable: co ; 1 L -^— I 

controllable but unobservable: co ; r~> l co K — 

uncontrollable but observable: co; „ t 



uncontrollable and unobservable: co . 
In case of a continuous time LTI system, there exists then 
a coordinate transformation z = Tx such that 
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Of course, a similar approach exists for discrete time systems. 
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The transfer matrix from u to y is given by 

G(s) = C co (sI-A co ) -1 B co . 

It is important to remember that, even though the transfer matrix of a dynamical 
system involves only its controllable and observable part as shown by this equa- 
tion, the internal behaviour, e.g. the response to nonzero initial conditions or to 
disturbances, is quite different. 

A triplet A, B, C, which is both controllable and observable and which satisfies 
G(s) = C(sl — A)~ B , is called a minimal realization of G(s) . 

An elegant way of determining the similitude transformation which yields the 
above decomposition has been given in [AlSa04] in Example 3.12. It relies again 
on the computation of the null spaces of the controllability and observability ma- 
trices by the MATLAB null command, or by their singular value decomposition 
(svd) if there is doubt about the plant's practical controllability and/or observabil- 
ity. The successive bases are obtained as follows: 

• uncontrollable and unobservable part: Nncno=null([Qc';Qo]); 

• uncontrollable but observable part: Nnco=null([Qc';Nncno']); 

• controllable but unobservable part: Ncno=null([Qo;Nncno']); 

• controllable and observable part: Nco=null([Nncno';Nnco';Ncno']). 

The similitude transformation is then: T=[Nco Ncno Nnco Nncno], and the ma- 
trices of the canonical decomposition are obtained by (A.30). 



A.5 Solving the State Equations in Time Domain 

The total response to an initial state x at t = and an input u(7) or u k is: 

continuous case: x(t) = e At x + f e A( '~ r) Bu(Y)fi?T . (A. 18) 

Jo 
t-i 

discrete case: x k = O* x +^O i_1 "'' Tu,. . (A. 19) 

i=0 

The proof of (A. 1 8) is similar to solving a scalar differential equation, by left 
multiplying (A. 7) by e~ , [Ost04]. A matrix exponential is defined by: 

e A =f2— = 1 + - + — +- , (A.20) 

t k\ 1! 2! 

and has the following property in a change of basis (see Sect. A. 7): 
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e T_lAT = T _1 e A T, VT invertible , (A.21) 

the proof of which is trivial by replacing A in (A.20) by T~ AT . (A.19) is estab- 
lished by repetitive application of (A. 8) from the initial state x until time step k. 

Application. Proof of (A. 9). 

By applying (A. 1 8) to the interval kT s <t<(k + l)T s , thus with the replacement 



B u(t) dr . (A.22) 



(A.23) 

and assume that the input u(t) = u k is constant for kT s <t <(k + Y)T S . By factor- 
izing the constant terms out of the integral, (A.22) can be written 



x = x k , we obtain 




X £+l = 


at; , f^ +1 > r < A(*+l)r s 
= e ■' x, + / e [ 


Introduce now 






O = e Ar < , 



<k+l)T s A ^ k+l)Ts 



Ox t + J e l ,s 'rfr-Buj. (A.24) 

J kT, 



The change of variable t' = (k + l)T s — t in the integral yields 

»(*+i)r s A [ (yt+1)7; _ T ] rO 



J kT s J T s Jo 

If A is invertible, this integral is evaluated as follows: 

/» T T 

S e AT dT = A- l e AT \ s =A~\e AT ° -I) = (O-FjA" 1 . (A.26) 

o lo 

(A.23) to (A.26) prove (A.9). 

Transition matrix (free response). 0(t) = e At , or e { - t ~ t °' ift ^0, respec- 
tively O k , is called transition matrix, since it produces the free transition of the 
system state from x at initial time t Q to x(t) at time t. 

For time varying systems, where the value of the initial time matters, one defines a 
transition matrix O(t,t ) , such that \(t) = O(t,t )\(t ) . An evident property is: 

<&{t 2 ,t l )<&{t l ,t{ ) ) = O(t 2 ,t ) , and another one is that O(t,t ) satisfies the system's 
homogeneous differential equation: O(t,t ) = A O(t,t ) . 
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A.6 Application of the Laplace or the z-Transform 



The transfer matrix of MIMO systems is obtained, as in the SISO case, by apply- 
ing to the state space representation the Laplace or the z-transform, as illustrated in 
Table A.5. 

Table A.5 Application of the Laplace or the z transform. 



Continuous case 



Discrete case 



Resolvent matrix: 

C[e At \ = C[0(t)] = (sl-A)- 1 Z[<b(kT)] = Zl<b k ] = (zi-<br l z 

Transfer matrix: 
Y(s) = G(s)V(s) Y(z) = GH»U(z) 

G(s) = C( 1 sI-Ar 1 B + D G(z) = C(zI-*) _1 r + D 

Case of SISO systems: transfer function: 



dct 



G(s)-- 



sI-A 



dct 



det(sI-A) 



G(z): 



zl O 



det(zI-O) 



Every pole of G(s) , respectively of G{z) , is necessarily an eigenvalue of A, 
respectively of O , (see Sect. A.7.4), but the inverse is not true (example: case of 
an eigenvalue which cancels also the numerator). 

Case of MIMO systems: poles and zeros 

The eigenvalues (or poles, by language abuse) of the plant (A. 7), respectively 
(A. 8), are as previously the roots of det(sl — A), respectively of det(zl — O). 

The zeros of this plant are the values of s, respectively of z, for which the equation 



(A -si B) 
C D 







(A.27) 



fo-zi r 

C D 



: (A.28) 



has a non trivial solution, thus, in the case where p = q , the values of s, respec- 
tively of z, solutions of 



A-iI B 
C D 



= 



(A.29) 



o-zi r 

C D 



= 



The first member matrix of (A.27) is also called Rosenbrock system matrix 
[Rop90]. If s = fi, respectively z~\i, is such a zero, the solution x z , respec- 
tively u z , of (A.27), respectively of (A.28), given by 
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A /il B 
C D 



0, 



o ^i r 

C D 



o, 



is called state vector direction, respectively control vector direction, associated 
with this zero. 



Remark A. 1. Dynamic characterization of a zero. If x =x z and if, for t>0. 



u(t) = u z e' 1 ' , then \(t) = \ z e' 1 ' , which makes that 



y(0 = Cx + Du = (Cx z +Du z )e M '=0, V?. 



A.7 Transformation to one of the Canonical Forms 

Due to the formal similitude of the two cases, the rules and transformation theo- 
rems which follow will be given only for the continuous case 



A. 7.1 Basis Change in the State Representations 

Assume that new coordinates (state variables) have been introduced by means of a 
transformation defined by a regular matrix T such that 

x = Tz , 

where x and z denote respectively the old and the new state vector. In the new ba- 
sis, the state equations become 

z= Az+Bu 
y = Cz + Du 

with: 

A = T~ 1 AT, B = T _1 B, C = CT, D = D. (A.30) 



Such a transformation is called a similitude transformation, the matrices A and A 
are called similar matrices. 

Characteristic equation invariance in this transformation. The characteristic 
equation det(A I — A) = is invariant in a change of basis defined by a regular 

transformation matrix T. A similitude transformation does not modify thus the ei- 
genvalues of a matrix. 
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A. 7.2 Transformation to Controllability Canonical Form 



The following rule and theorem apply to single input systems. 

Rule A.l. In the controllability canonical form, the last row of the system matrix 
is composed of the coefficients of the characteristic equation (except the one of 
the highest power term), with opposite signs. 

Theorem A.l. If the system x = Ax + b« (A and b constant) is controllable, it 



is possible, by means of the transformation z = Vx 
trollability canonical form 



- T x to put it in the con- 



where: 



z = A c z + b c M , 



A C =VAV" 



f 1 



C/a t/i 



o ) 



l n-l) 



, b c =Vb 



(A.31) 



and where the inverse transformation matrix is given by 



V = 



T 

q c 

qlA 



lie A ) 



(A.32) 



where q c is the last row of the inverse controllability matrix, Q c , and is there- 
fore deduced, according to (A. 12), from the following system of equations: 



qjb 




= 


qjAb 




= 


qjA"" 


2 b 


= 


qjA"" 


'b 


= 1 



(A.33) 



i.e. also 



qjQ c =(0 - 1) 
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A.7.3 Transformation to Observability Canonical Form 

The following rule and theorem apply to single output systems. 

Rule A.2. In the observability canonical form, the last column of the system ma- 
trix is composed of the coefficients of the characteristic equation (except the one 
of the highest power term), with opposite signs. 

[x = Ax + bw T 

Theorem A.2. If the system \ (A, b, c constant) is observable, it 

[y = c x 

is possible, by means of the transformation x = Tz, to put it in the observability 

canonical form 



where 



T"AT 



z = A n z + b n u 



y = <?o J - 



1 -«„-i 



c 1 T = (0 



1), (A.34) 



and where the transformation matrix is given by 

T = (q Aq 



A" _1 q 



O ' 



(A.35) 



where q is the last column of the inverse observability matrix, Q , and is 
therefore deduced, according to (A. 16), from the following system of equations: 



c T q 

c T Aq 



= 
= 



(A. 3 6) 



c T A"" 2 q = 



i.e. also: 



Qo <\o = (° 



l) 1 
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A. 7.4 Transformation to Diagonal Form 



If the eigenvalues of the (n x n) matrix A are all distinct, the system 
x = Ax + Bu 



y=Cx+Du 



can be represented in the new basis by the equations 



|x*=Ax*+Bu 

[y = Cx*+Du 



(A.37) 



where, due to their particular importance, we have denoted by x* instead of z the 
new state vector coordinates, also called modal coordinates, and where 



(A, 



A = T ! AT = A 



) 



A. 



(A.38) 



the matrices B, C and D being given by (A.30). 

The eigenvalues X 1 ,...,X n of A and its eigenvectors v l ,...,\ n are defined by 

the relation Av, = A,v, , thus also 



(A,I-A)v,.=0. 



(A.39) 



Solving the characteristic equation of A, i.e. 

det(A,I-A) = 0, 



(A.40) 



yields the eigenvalues A, , to which correspond thus the non trivial solutions v ; of 

(A.39), i.e. the eigenvectors of A. 

The new basis is composed of these eigenvectors, and the transformation ma- 
trix T, called here modal matrix, is given by 



«ii v.. 



v l ¥ 2 



(A.41) 



V i U i 

n\ «2 
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The left eigenvectors of A, by opposition to the above eigenvectors or right ei- 
genvectors, are the vectors w ; which satisfy 



W^A = A;W^, 



(A.42) 



thus also 



w/(A ; I-A) = 0. 



(A.43) 



It should be noted that the rows of the inverse transformation matrix T l are 
equal, apart from a multiplying factor, to the transposes of the left eigenvectors, 
w, T . Indeed, (A.38) can also be written T _1 A = AT -1 , therefore, denoting by t ; T 
the ;-th row of T~ , 



'f 






V 


/ 




A = 


A 




= 


A. 






fn. 





A, ••• 0) 



••• A, 



C*t 



The i-th row of this matrix equation, t, T A = A, t ; , shows with (A.42) that 



t, T = wJ. 



(A.44) 



Remark A. 2. The left eigenvectors of a matrix A are right eigenvectors of its 
transpose, since (A.43) implies that (A,! — A T )w, =0 . 



A.7. 5 Transient Response of a System as a Function of 
its Eigenmodes 

The transient response of a linear time-invariant system to an initial condition 
\(t ) = x and an input excitation u(t) , given by (A. 18), involves the matrix ex- 
ponential e ' , which can be expressed in a particularly simple way by using the 
diagonal representation. Indeed, thanks to (A.21) and (A.38) we can also write 



At T T-'A^Trp-l rp A^rp-1 

e = Te T = Te T . 
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„A< 



Since A? is also a diagonal matrix, e is it too and is composed of the exponen- 
tials of the diagonal elements of At . 

Involving the right and left eigenvectors of A, thus the columns of T and the 
rows of T" 1 , we can write 



At I 

e = v, 



X,t 







\„t 



( Tl 
W, 



w 



EXit T 



which yields, from (A. 1 8), the following system response 



' eigenmode scalar 
product 



fi +£v,.jV' ( ' T) yvjB»(T)dr. (A.45) 



This expression reveals the system eigenmodes, composed of an eigendirection 
V, and a factor, which decays exponentially with a time constant equal to the in- 
verse absolute value of the eigenvalue. These modes are involved in both the free 
response and the forced response, where right and left eigenvectors play comple- 
mentary roles [DucOl]. More precisely, 

• the eigenvector v, determines the distribution of the corresponding mode e ' 
on the components of the state vector: if e.g. the y'-th component of v ; van- 
ishes, the mode e ' will not appear on the y'-th component of \(t) ; 

• the transpose left eigenvector w, T determines the way the mode e ' is excited 

by the different components of the initial state x and the row vector w^B the 
way this mode is excited by the different components of the input u(t) . Denot- 
ing e.g. by c ; the scalar product w/xq , the initial state x , as deduced from 

(A.45) written for t = t = , can be written x = ^c,v. . 



The c ; 's are thus nothing else than the components of x in the eigenvector 
basis. In particular, 



\;t 



1 . if x is collinear to v • , only the mode e ' will appear in the free response; 

2. if x is orthogonal to w ; , the mode e ' will not appear in the free response 



B Complements of Matrix Calculus 



B.l Trace of a Matrix 

Definition B.l. If A is a square matrix of size (nxn) , the trace of A, denoted 
tr(A) , is the sum of its diagonal elements: 

A = (a (> ) => tr(A) = ]T>„. . 

Property B.l. If A is an (n x m) matrix and B an (m x n) matrix, 

tr(AB) = tr(BA). (B.l) 

n n m m n m 

Proof: tr(AB) = £(AB), = ££aA = ££ Va = E( BA )« = tr ( BA ) ■ 

i=l i=l it=l k=\ i=l k=\ 

Special case: m = 1. If a and b are two vectors of dimension «, the trace of a sca- 
lar being this scalar itself, 

tr(ab T ) = b T a. (B.2) 

Application. By letting a = Mx , where M is an (n x n) matrix, and b = x of 
dimension n, 

x T Mx = tr(Mx-x T ). 

B.2 Matrix Inverses 

After recalling the definition and calculation of the inverse of a manix, a few use- 
ful inversion formulas will be given. 
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B.2.1 Inversion of a Matrix 

If A is a square matrix, with det(A) ^ , it admits an inverse defined by 
A^ A = I and calculated by 

A" 1 - ^^ ad J( A )> with (*lj(A)) w = (-l)' +i det(A y; ) , 

where A„ is the matrix obtained by deleting the i row and the/ column from A. 
If A -1 exists, A is said invertible or regular, and det(A _1 ) = l/det(A) . 

Unitary matrix. A real matrix U is called unitary, if U T = U~ . Thus U T U = I . 

B.2.2 Matrix Inversion Lemma 

If the matrices A, C and A+ BCD are invertible, 

(A + BCD)" 1 =A" 1 -A" 1 B(C" 1 +DA" 1 B)" 1 DA _1 . (B.3) 

Proof. Let us left multiply the two members of this equation by A+ BCD : 

I = I + BCDA- 1 -B(C" 1 + DA- 1 B)- 1 DA- 1 -BCDA- 1 B(C" 1 + DA _1 B)- 1 DA- 1 
= I + BCDA ' B(I + CDA "BXC" 1 + DA _1 B) ' DA" 1 

By left factorizing BC in the third term of the second member, we get for it 

I + BCDA" 1 BC(C _1 + DA" 1 B)(C" 1 +DA" 1 B)" 1 DA" 1 
= I + BCDA "' BCDA" 1 = I . 



B.2.3 Inverses of Some Particular Partitioned Matrices 

First case: if the matrices A and D are invertible, 

D" 1 ) 

(B.4) 



A B 
D 



A" 1 -A _1 BD _1 



D 1 
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Proof. By verification that the product of this matrix with its non-inverted original 
is the identity matrix, knowing that the inverse of a matrix is unique. 

Second case: if the matrices B and C are invertible, 



A B 
C 



B 1 B "AC" 1 



(B.5) 



Proof. Similar to the previous case. 



B.3 Matrix Analysis 



B.S.I Differentiation of a Scalar with Respect to a Vector 
(Gradient) 

Definition B.2. \fy is a scalar function of n variables x i , considered as the com- 
ponents of a column vector x (respectively of a row vector x T ), the expression 



dy_ 

dx 



■ . dy 
respectively — — 
dx 



denotes a column vector (respectively a row vector) of components dy/dx t : 

' dy s 



dy 

dx 



dx. 



8y 



dx„ 



= Vj, 



(B.6) 



dy 
dx 1 



dy dy 



dx. 



dx„ 



dy 



dx 



(B.7) 



Property B.2: If a and x are vectors of dimension n, 



d l T x d T 

— (a x) = — (x a) = a. 
dx dx 



(B.8) 
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Proof. For the first part of this equation, 



B Complements of Matrix Calculus 



d . T . d . . 

— *) = — («i*i + ... +a n x n )-- 
dx dx 



Furthermore, the transpose of a scalar being this scalar itself, 
x T a = (x T -a) T = a T -x , which proves the second half of (B. 8). 



Property B.3: 



— (x T Ax) = (A + A T )x. 
dx 



(B.9) 



Proof. By comparison with the scalar situation, where 



d du dv d d 

-r( UV )=-r V + U -T = -r (" V )v=constant + "T" ( MV )„=constant . 

dx dx dx dx dx 



the following applies here: 



A (x T Ax) = A (x T a) 

dx dx 



^(b T x) 
dx 



= Ax + A x. 



Remark B. 1. If A is symmetric, 



— (x T Ax) = 2Ax. 
dx 



(B.10) 



Special case A = I : 



— (x T x) = 2x. 
dx 



(B.H) 



B.3.2 Differentiation of a Vector with Respect to a Vector 



Definition B.3. If y is an (mxl) vector, the elements of which are functions of n 
variables x i , considered as the components of a (1 x ri) row vector, x T , the ex- 
pression dy/dx T denotes an (mxn) matrix, with elements dyJdx: : 
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Property B.4: 





fyi 




dy x 


dy 


dx l dx n 


dx T 


d y m dy« 




dx x dx n 


dy T _ 


'dy/ 




dy 


dx 


dXj 




dx 1 



(B.12) 



(B.13) 



Remark B. 2. If x and y have the same dimension, det(dy/dx J ) is called Jaco- 
bian determinant, or simply Jacobian, of y. 



Property B.5: 



^ T (Ax) = ^-(x T A) = A. 



dx 



dx 



(B.14) 



Proof. Let y = Ax , with A G R mxn . According to the definition equation (B.12), 



dy 


d 




■ + a l„ X n 




«n • 


•• «1« 


dx 1 


dx 1 


, a ml X l+ ■ 


. + a x 

1 mn n ) 




,«ml • 


a 

mn } 



:A. 



For the second half of the relation, it holds that 



f(x T A) = 
dx 



dx 



"(x T A) T = 



dx 



-(A T x) =(A T ) T . 



Remark B.3. Special case A = I : 

dx 



dx' 



dx 1 dx 



(B.15) 



Property B.6: Case of a dot product containing a vector function of x. Con- 
sider the vectors x, of dimension n, and y, of dimension m, function of x, and the 
constant vector a of dimension m. The following differentiation formulae hold: 



d , i . d , T N <fy 
— (a T y) = — (y T a) = -/-a. 
ax ax ax 



(B.16) 
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Proof. Since a T y is a scalar, we have, according to (B.6), 



d , t ^ d , s 

—(a y) = —(a l y l + ... +a m y m ) = 
dx dx 



dx 1 

dy\ 

dx„ 



dy m ) 

dx x 

dy m 

dx„ 



The obtained expression can also be written as 



d i t n 

-i-(* y) : 

ax 



dy x 
dx x 


9y m 

dx x 


( \ 


dy x 


dy m 

dx n, 


a 

V m ) 



df_ 

dx 



according to (B.12) defining the derivative of a vector with respect to a vector. 
Furthermore, y T a = a T y , which proves the second half of (B. 16). 



B.3.3 Differentiation of a Scalar with Respect to a Matrix 



Definition B.4. If y is a scalar function of mxn variables x ; , considered as the 
elements of a matrix X, the expression dy/dX denotes a matrix, with elements 
dy/dXij ■ 



dy 



dy 



dy _ 


dx n 


dx ln 


dX 


dy 


dy 




y dx ml 


dx mn. 



(B.17) 



The following properties of this subsection will hold only in the case X is a square 
matrix, of size (nxn) . 



Property B.7: 



dX 



tr(X) = I. 



(B.li 



B.4 Linear Algebra 










Proof: 








'1 o • 


— tr(X) = 
dX 


d 

' dX 


\ i ) 


= 


1 






v o ... • 


Property B.8: 


— tr(AX T ) 
dX 


= A. 
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••• 0) 



... 1 



(B.19) 



Proof: 



jU (ax T)=A 

dX dX 



Y,Y, a ik x ik 



\"n\ 



= A. 



Property B.9: 



^tr(XAX T ) = X(A+A T ). 



(B.20) 



Proof. Following a similar approach to the proof of (B.9), we can write 



dX 



tr(XAX'): 



dX 



tr 



X- (AX 1 ) - 
constant] 



dX 



tr 



(XA) -X 1 . 
constant ! 



Using (B.19) and its transpose, the previous expression becomes 
d 



dX 



tr(XAX T ) = (AX T ) T + XA = XA T + XA . 



Remark B.4. If A is symmetric, 

d 
dX 



tr(XAX T ) = 2XA. 



(B.21) 



B.4 Linear Algebra 



Here only main results will be presented. For more details the reader is referred to 
textbooks such as [MeyOl] for basic Linear Algebra, or [ZhDG96] for more spe- 
cific properties used in the Control Systems context. 
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Let some matrix A £ R mx " be a linear transformation from the vector space R" 
to the vector space R m . 

Definition B.5. The column space C(A) of A, sometimes also called the image 
or range of A, is the set of all linear combinations of the columns of A. It is there- 
fore a subspace of R m , said to be spanned by the columns of A. It can be consid- 
ered also as the following set: 



C(A) := {y E R m : y = Ax, xel"). 



Definition B.6. The null space N(A) of A, sometimes also called the kernel 
Ker(A) of A, is the set of all solutions of Ax = , i.e. 



N(A) := {x e R" : Ax = o} . 



It is therefore a subspace of R" . 

Definition B.7. The orthogonal complement of a subspace S C R" is defined by 



S L := {y G R" : y T x = for all x £ s} . 

Property B.10. It is easy to show that dim[C(A)] + dim[A^(A)] = n , where the 
dimension of a subspace S, noted dim(S) , is the size of a basis of this subspace. 
Moreover, the following holds: 

A^(A) = [C(A T )] ± <£> N(A)_LC(A T ), 

in other words, the null space of A is the set of vectors orthogonal to all rows of A, 
which are the columns of A T . 

Definition B.8. The system of m linear equations Ax = y in the n unknowns 

Xj , . . . , x n , the components of x, has a solution if and only if y € C(A) . 

If A is square, of size (nxn) , this solution is unique if and only if rank(A) = n . 

Definition B.9. The rank of A, denoted rank(A) , is the size of the largest non 
vanishing determinant contained in A. It is therefore equal to the maximal number 
of independent rows or columns, i.e., 

rank(A) = dim[C(A)]. 
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B.5 Singular Value Decomposition 

An interesting tool in matrix analysis, which is very useful in systems analysis and 
control in the MIMO case, is the Singular Value Decomposition (SVD). 



Theorem B.l. Let AG : 



There exist unitary matrices 



U = (uj u 2 
V = (v, v 2 



u„ e. 



vje. 



U'U = I„ 
V T V = I„ 



such that 



A = USV' 



fs. 



>px{n-p) 



(m-p)xp (m— p)x(n— p) / 



(B.22) 



where 



S 1 = 



'o\ 


• 


• o s 





<T 2 • 


• 





• 


• V 



and 



°"i > °~2 — " ' — a P — 0' P = mm { m > n } ■ 



The <t, are the singular values of A. The vectors u ; and v ; , called respectively 
left singular vectors and right singular vectors of A, satisfy respectively 



AV, = C7;U; 
AT «, = ^i ■ 



Connection with eigenvalues and eigenvectors. The above equations can be 
written as 



A T Av ; =a, 2 v,. 

AA T u,.=a, 2 u, 



Thus, erf is an eigenvalue of AA T and A T A , and u ( , respectively v, , are ei- 
genvectors of A A T , respectively A T A . 
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Maximum and minimum singular values. It is common practice to denote 

<j( A) = <Tj = the largest singular value of A 
<r(A) = a = the smallest singular value of A 

It can be shown that, for any eigenvector v, of A T A , we have llAvJI = <r, |v,|| , 
where || • | denotes the Euclidian norm of a vector, i.e. its geometric length 

yV[ +v 2 -\ \-v n . Hence the following interesting interpretation of singular 

vectors: 

• Vj (\ n ) is the input, i.e. controlling direction with the highest (lowest) gain; 

• Uj (u m ) is the output, i.e. observing direction with the highest (lowest) gain. 

If A is square and invertible, (B.22) and the unitary property of U and V yield 

A-^VS-'U 1 , 

which implies that the singular values of A" are the inverses of those of A. In 
particular, 

a(A" 1 ) = -i- (B.23) 

2(A) 

Condition number. The condition number of A is defined as 

7(A) = ^>1. 

According to this definition and to (B.23), 7(A _1 ) = 7(A) . 

When the condition number is very large, A is said ill conditioned. A small 
change of one of its elements can change its rank (see below). 

Connection with the rank of a matrix: lemma B.l. Let A £ M. mxn and 

P\>o- 2 >--->o- r > <7 r+1 = • • • = 0, r < min \m, «} 

be its strictly positive singular values. 

Then rank(A) = r , -/V(A) is spanned by {v r+1 ,...,v n } and C(A T ) = [N(A)] is 



spanned by {v 1 ,...,v r }. 
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As mentioned by several authors, e.g. by [AlSa04], this is a safer way to test the 
rank of a matrix than Definition B.9. A very large condition number, indeed, may 
mean that the smallest singular value is so close to zero that the practical rank of 
A is r — 1 instead of r. A striking example is given in [AlSa04], Appendix B. 

Reduced singular value decomposition. Since S contains mainly zeros, a re- 
duced SVD is sometimes introduced, in which most of these zeros are eliminated. 
Theorem B.2. Let A <G R'" x ", with rank(A) = r . Then there exist unitary matri- 



ces U, G M'" 


<r and Vj € M. nxr such that 




A=U,IV I T , 


where 






E = diag(cr 1 ,...,(T r ), 


and 






°~\ > °~2 — ' ' ' — °V > 



Uj , respectively Vj , is obtained from U, respectively V, by dropping its rightmost 
(jn — r), respectively (n — r), columns. 



B.6 Positive Definite and Positive Semidefinite 
Matrices 

Consider a matrix A of size n x n and symmetric. 

Definition B.10. A is called positive definite if for any vector x * the quadratic 
form x T Ax is positive. 

Definition B.ll. A is called positive semidefinite (or nonnegative definite) if for 
any vector x ^ the quadratic form x T Ax is positive or equal to zero. 

Criterion of positive definiteness (Sylvester criterion). A matrix A is positive 
definite, if and only if the determinants associated with all upper-left submatrices 
of A are positive, i.e. 



a u >0; 



>0; 



>0; 



'22 



>0. 
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Criterion of positive semidefiniteness. A matrix A is positive semidefinite, if 
and only if the determinants associated with all submatrices of A, the diagonals of 
which belong to the diagonal of A, are positive or equal to zero, i.e., as illustrated 
here in the case of a 3x3 matrix: 



i n , a 22 , £33 > ; 



'22 



'23 



>0; 



''23 



"M3 



a 22 a 23 
a-,, a 



'33 



>o. 



Equivalent criterion for symmetric matrices. A symmetric matrix A is positive 
definite if and only if all its eigenvalues are positive; it is positive semidefinite if 
and only if all its eigenvalues are positive or equal to zero (nonnegative). 

PropertyB.il. Given an mxn matrix D, the symmetric nxn matrix A = D T D 
is positive semidefinite. It is positive definite if and only if D is of rank n. 

Proof. Consider the vector y defined by y = Dx. Then the following holds: 
x T Ax = x T D T Dx = y y = Y^J, > . The inequality of the last member of this 
expression becomes an equality if and only if y = , thus D x = , which is im- 
possible for x v£ if D is of rank n. 

Square root of a matrix. Given a matrix D and a matrix A as defined in Property 
B.l 1, D is called a square root of A. If D is a square root having a number of rows 
equal to the rank of A, all other square roots are obtained by DT, where T is any 
matrix for which TT T = I . 

If A is positive semidefinite, there exists a matrix A 1 ' 2 which is a symmetric 
square root of A, and which is itself positive semidefinite. It satisfies 
A = A 1 ' 2 A 1 ' 2 . Moreover, if A is regular, so is A 1 ' 2 . 

Other properties: 

• If A and B are positive semidefinite, so is A+ B . 

• If one of them is positive definite, so is A + B . 

• If A is an n x n positive semidefinite matrix and if B is an mxn matrix, then 
BAB T is positive semidefinite. 
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C.l Probability Theory and Random Processes in the 
Scalar Case 

This appendix is a summary of the main concepts. For a detailed presentation, re- 
fer to text books, such as [Kay06], [ShBr88] or [BrSi75a and b]. 



C.l.l Random Variable 



C.l. 1.1 Definitions 

A random variable X is a quantity whose value x depends on the outcome of a ran- 
dom experiment. A random variable is said continuous if it takes its values x in 
]— °o ; +oo[ ; it is said discrete if it can take a finite number, or a countably infinite 
number, of real distinct values: x , x, , . . . . 



C.l. 1.2 Probability Distributions and Main Characteristics 

The basic concepts and properties of a single random variable are summarized in 
Table C. 1 for both the continuous and the discrete situations. 

The case of two or of n random variables (random vectors) will be discussed in 
subsequent sections. 
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Table C.l Main concepts of random distributions for the continuous and discrete cases. 

Continuous Case Discrete Case 

Distribution function: F(x) = P\X <x\ 

Probability density function: p(x) : Probability mass function: p k : 

p(x)dx = P{x<X<x + dx} p k =P{X = x k } 

p(x)>0, f p(x)dx = l Pk>^2_ J Pk= l 

Expected value or (mathematical) expectation, or (statistical) mean, or 1 st order moment: 

/i = E{X} = f xp(x)dx ! J = E W = Eft x * 

Expectation of the random variable Y = g(X) , where g(x) is a real function of the variable x: 

E{gW} = f°° g(x)p(x)dx E{g(X)} = J2P* g( x t) 

Moment of order n = expectation of g(X) = X" : m — EJ X " \ 

m n = f x"p(x)dx m n =^p k x" k 

J -°° k 

Centered moment of order n: E I (X — fj,)" \ 

Variance of X, or centered moment of second order: 

Var{X} = E{(X-/i) 2 } (C.l) 

a x , or simply a if there is no ambiguity, denotes the standard deviation of X, which is equal to 
the square root of the variance: a\ = Var{X} . 

C.l. 1.3 Two Random Variables 

The previous notions extend themselves easily to a set of two random variables, X 
and Y, defined with respect to the same experiment. Let us consider here only the 
case where they are continuous, and define \x x — E [X } and \x Y = E {7} . 

Bivariate distribution function: 

F XY (x,y) = P{X<x and Y < y) . (C.2) 

Joint probability density: function of two variables p(x,y) such that 

p(x, y)dxdy — P{x < X < x + dx, y<Y<y + dy} , (C.3) 
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with p(x,y)>0, I I p(x,y)dxdy = \ . 

Kf -ocj — oo 

Expectation of a function of X and Y, i.e. g(X,Y) : 

E{g(X,Y)}= I I g(x,y)p(x,y)dxdy . 

*J —doc/ — oo 

Mixed moments: let g(x,y) = x r y s , where r and s are positive integers: 

m rs =E\x r Y s }= r r 'x'y s p(x,y)dxdy. (C.4) 

Central mixed moments: 

E{(X-p x Y(Y-p Y Y}. (C.5) 

Covariance: it is the centered moment of second order: 

Cov{X,Y} = E{(X-p x )(Y-p Y )} = E{XY}-p x p Y . (C.6) 

Correlation coefficient or simply correlation: 

Cov{X,Y} Cov{X,Y} 

p(X,Y) = -= l J = ^-^-. (C.7) 

^/Var{X}Var{y} v x a r 

Definitions C.l. Two random variables X and Y are said 

• uncorrected if Cov{X,Y} = 0; then, (see (C.6)), E{XY} = E{X}~E{Y} ; 

• orthogonal if E {XY} = ; 

• (statistically) independent if p(x, y) = p(x)p( y) . 

If X and Y are independent, then, according to (C.4): E{XY} = E{X}E{Y} . 
Consequently, according to (C.6): Cov{X,7} = 0, i.e. the random variables X 
and Y axe uncorrected. 
Conclusion: two independent random variables are thus always uncorrelated. 

However, the inverse is not true. Two uncorrelated random variables are not 
necessarily independent, except if they have Gaussian distributions. 
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C.1.2 Random Function. Random, or Stochastic Process 

Definition C.2. A function X(t) of the independent variable t is a random func- 
tion when, for any value of the independent variable, its value is random. A ran- 
dom (or stochastic) process characterizes the evolution, as a function of some pa- 
rameter t, of a system whose behavior is the result of chance. 

In the majority of situations, but not always, the considered parameter t is the 
time, either continuous, or discrete. 

In practice, the random functions appear often as signals which are logged as 
functions of time. In the following, we will consider the set £ N of the values of a 

random function X(t) , defined by N records x t {t) (N realizations of the random 
experiment), where N can be finite, countably or uncountably infinite. Such a re- 
cord x t (t) is also called a sample, or a member function. 



C.1.3 Continuous-time Random Processes 
C. 1.3.1 Ensemble Characteristics of First Order 

The value at some time t of the various samples is a random variable X(t) , from 
which are defined the following characteristics of the random process: 

Distribution function: F(x,t) = P\X(t) < x} . 

Probability density: function p(x,t) such that p{x,t)dx = P\x <X(t)< x + dx] 

Process mean: p x (t) = E |X(/)} . 

Process variance: Vav{X(t)} = EJ [X(t)-E{X(t)}} 



C. 1.3.2 Ensemble Characteristics of Second Order 

The observation of the various samples at two distinct times, t x and t 2 , constitutes 
a two-dimensional random variable [X(^), X(7 2 )] • 

Distribution function: F(x l ,x 2 ,t x ,t 2 ) = P{X(t x )< x x and X(t 2 )< x 2 } . 
Joint probability density: function p(x l ,x 2 ,t 1 ,t 2 ) such that 

p{x x ,x 2 ,t l ,t 2 ) dx x dx 2 = P {xj < X(t x ) < Xj + dx x and x 2 < X(t 2 ) < x 2 + dx 2 } . 
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C. 1.3.3 Stationarity 

A stochastic process is said stationary of second order, or wide sense stationary, if 
its first and second order ensemble characteristics do not depend on the origin of 
time. In particular, its ensemble mean is constant: \x x (t) = /i x . 

The main second order ensemble characteristics are collected in Table C.2, in the 
case of a single stochastic process X(t) , on the left side, and of two distinct sto- 
chastic processes, X{t) and Y(t) , on the right side. In the case of stationary proc- 
esses, these characteristics depend only on the difference between the two instants, 
t 2 -t l =T. 

Table C.2 Second order ensemble characteristics of one and two continuous random processes. 

One process: X(t) Two processes: X(t) and Y{t) 

1. General case (arbitrary processes): 

Autocovariance function: Cross-covariance function: 

r xx (t 2 ,t 1 ) = Cov{X(t 2 ),X(t 1 )} r XY (t 2 ,t l ) = Cov{X(t 2 ),Y(t l )} 

= E{[X(t 2 )-fi x (t 2 )][X(t l )-n x (t l )]} =E{[X(t 2 )-n x (t 2 )][Y(t l )-Mh)]} 

Autocorrelation function: Cross-correlation function: 

i Pxx {t 2 ,t l ) = E{X{t 2 )X(t l )} tp xr (t 2 ,t i ) = E{X(t 2 )Y(t l )} 

2. Stationary processes: t 2 —t i = T 

Autocovariance function: Cross-covariance function: 

r xx (T) = Cov{X(t + T),X(t)} r xr (r) = Cov{X(t + r),Y(t)} 

Autocorrelation function: Cross-correlation function: 

Vxx (r) = E{X(t + r)X(t)} VxY (r) = E{X(t + r)Y(t)} 

3. Centered stationary processes: (i x =0, fi Y =0 

r xx( T ) = <Pxx(i~) r XY (r) = ip XY {T) 

Autocovariance = autocorrelation Cross-covariance = cross-correlation. 



C.l.3.5 Temporal Averages and Ergodicity 

In practical situations, it is not easy to calculate ensemble characteristics, which 
would require running simultaneously the same experiment on many copies of the 
plant under investigation. It is much easier to experiment with one single sample 
record x t (f) of the stochastic process, and to determine time averages calculated 
on this sample on a finite time horizon. We define therefore the 
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Temporal average of one sample x t (t) : 



(x i (t)) T =— J x,(t)dt. 



Although this time average is constant for a given sample, the set of corresponding 
values, if we could have taken them over all the samples, is a random variable, 
(X(/)) r , of which (x t (t)) T is one realization. From a practical point of view, it 

would be nice if we could estimate the ensemble mean of the stochastic process 
X(t) from just one of its sample functions. 

Now, for stationary processes, the ensemble average [i x (t) is a constant fi x . 

In this case, if 

lim E{(X(t)) T } = n x and lim Var {{X(f)) T } = , 

then the time -averaged mean and the ensemble mean are equal for T — > oo . This 
property does not hold for all stationary processes, but only for a special (often 
encountered) class of random processes, the ergodic random processes. 

A stationary stochastic process is said ergodic, when its time average and its 
ensemble mean are equal. Of course, in that case, the time average, at the 
limit T — > oo , does not depend anymore on the sample x i (t) chosen. 



C.l.3.6 Power Spectral Density (PSD) 

Definition C.3. The power spectral density of a stationary random process X(t) 

is the Fourier transform of the autocorrelation function ip xx (r) of this random 

process. 

Thus, by adopting the symbolic writing f(t) <-> F{u) = ^~[f(t)] to denote a pair 

composed of a function of time and its Fourier transform, we can write: 

1 r°° ■ , r°° _ ■ 

l Pxx( T ) = T- | ^ xx(oj) e JUJT duj <-► <S> xx (u)) = / ipxx(r)e JUT dT, 
2irJ-°° J -~ 

and, for two processes: 



^at( t ) = — <&xY(u)e JU}T dw <-> $ xy (w)= ipxr(T)e JiJjT dr. 
N.B.: ^^(u;) is a real function of to because ^xr(r) is an even function of r . 
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C.1.4 Discrete-time Random Processes 



As in the continuous case, the first and second order ensemble characteristics of a 
discrete stationary stochastic process do not depend on the origin of time (see Ta- 
ble C.3). In particular, its ensemble mean is constant: /J, x (k) = /i x . 

Table C.3 Second order ensemble characteristics of one and two discrete random processes. 



One process: X(k) 



Two processes: X(k) and Y(k) 



1. General case (arbitrary processes): 

Autocovariance function: 

rxx (j,k) = Cov{X(j),X(k)} 

= E{[X(j)-fi x (j)][X(k)-^ x (k)}} 

Autocorrelation function: 

Vxx (j,k) = E{X(j)X(k)} 



Cross-covariance function: 

r xr O\k) = Cov{XU),Y(k)} 

= E{[XU)-H x U)][Y(k)-n r (kj\} 

Cross-correlation function: 

VxY (j,k) = E{X(j)Y(k)} 



2. Stationary processes: j — k = m 
Autocovariance function: 

r xx (m) = Cov{X(k + m),X(k)} 
Autocorrelation function: 

Vxx (m) = E{X(k + m)X{ky] 



Cross-covariance function: 

TxY ( m ) = Cov {X(k + m), Y{ky] 
Cross-correlation function: 

ip^im) = E{X{k + m)Y{ky] 



3. Centered stationary processes: fi x = ; jj, Y = 
r xx (m) = ifi xx (m) 







r XY (m) = ifi XY (m) 



4. Power spectral density: 

<fixxW = —- \ ^xx^eJ^du 

lit J -TT 



*„(») = Y,VxY(k)e- jku 



C.2 Random Vectors and Vectorial Random Processes 



C.2.1 Definitions 



The case of two random variables X and Y, seen in Sect. C. 1.1.3, could have been 
considered as the situation of a two-dimensional random variable (X ,Y) . 



326 C Review of Probability Theory 

The generalization to n dimensions follows then quite logically. 

Random vector. An «-dimensional random variable (X l ,X 2 ,...,X n ) is often 
considered as a random vector 

x = {x l x 2 ... x n )\ 

the components X i of which are (scalar) random variables. 

If the X i are continuous random variables, X is a continuous random vector. 

This is the case considered here for the introduction of the new concepts, specific 
to the vector aspect of the situation. The corresponding definitions in the case of a 
discrete random vector will be deduced easily from that case by analogy. 

Probability density. One calls probability density of the random vector X the 
function 



such that 



and that 



p(x 1 ,x 2 ,...,x n ) = p(\), 
P{\ < X < x + dx} = p(x) dx . 



/p(x)dx = I ••• I p(x l ,x 2 ,...,x n )dx l dx 2 ---dx„ =1. 
-CO \J — CO \J —CO 

Mean or expectation of the random vector X: 

i i x =E{X}= T xp(x)dx = (E{X l } - E{X n })\ 

%J -co 

which is a vector. 

Variance of the random vector X, and corresponding covariance matrix: 

E^ =Var{X} = E{(X-^ x )(X-^) T } 

'VarjXj Cav{X u X 2 } ■■■ Cav{X u X H Y 

Cov{X 2 ,X,} Var{X 2 } •■• Cow{X 2 ,X n } 



(C.8) 



CovjX^X,} 



Var{X K } 



(C.9) 



Remark CI. The variance of a random vector X is thus a matrix. Since its off- 
diagonal elements are covariances between the scalar coordinates of the vector, 
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and even the diagonal ones by considering that Var{X ; } = Cov{X ( ,X ( }, this 

matrix is usually called a covariance matrix. Since this might lead to confusion, 
some authors call it a variance-covariance matrix. For the sake of clarity, we will 
speak in this book of the variance of a vector, e.g. Var{X} , and of its covariance 

matrix, e.g. E^ , even though these two appellations cover the same mathemati- 
cal concept. 

Properties of the covariance matrix: 

• ^xx i s an ( n x n ) ' symmetric matrix: 

^xx=^xx ■ (CIO) 

• When this matrix is diagonal, the (random) components X i of the vector X are 
uncorrelated. If, further, the probability density is factorable, as 

p(\) = p(x- l )p(x 2 )...p(x n ), 

the components X i are independent (see Sect. C. 1.1.3). 

• The covariance matrix of a random vector is always positive semidefinite. 

Proof. Consider a random vector X of dimension n and its covariance matrix 
Ejq. = Var{X} . Since this matrix is symmetric, let us associate with it the fol- 
lowing quadratic form, where u is a deterministic vector of same dimension as 
X: 

u T Z x ,u = u T E{(X-^)(X-^) T }u = E{u T (X-j J L ;r )(X-^) T u}. 
Since a T b = b T a for all vectors a and b, the following holds: 

u T Z x ,u = EJ[u T (X-^)] 2 

fr ,21 

= E \\u 1 (X l -n x )+ ... + u n (X n -p x ) 



Since the expectation of a square can only be positive or zero, it follows readily 
that u T Z xy u > whatever u. The quadratic form can vanish when u = or 
when 

u i( x i-»x,) + ••• +^(^-^1,) = ". 
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i.e. when there exists a stochastic dependence between the components of the 
random vector X (see proof of the next property). 

Therefore the quadratic form u T Z XY u , and thus also X^ , are positive 

semidefinite. 

• The covariance matrix of a random vector whose components are statistically 
independent is always positive definite. 

Proof. Let us develop the previous quadratic form: 

u T Z^u = J2 U ? yar{X i } + 2j2u i u J Cav{x i ,X J }. 

i 'J 

Since the X i are statistically independent, Cov|X ; ,X y - j = , as seen in Sect 
C.l.1.3. Consequently, u T Xu can only vanish if u i =0 V? , thus if u = . 
Covariance of two random vectors X and Y: 



^XY 



:Cov{X,Y} = e{(X-^)(Y-^) T }. (CM) 



T 



Property: ~L XY = 'L l YX . (C.12) 



C.2.2 Gaussian Random Vectors 

Definition C.4. A random vector X, of dimension n, is Gaussian if it has a prob- 
ability density of the form 

p(*)=— J — - e -{(*-M T ^(*-M (C13) 

(2^) f |^|2 

where \i x =E{X}, E^ = Var{X} and |£_ Kr | is the determinant of ~Lxx . 

In the Gaussian case, the probability distribution of the vector X is thus com- 
pletely determined by \i x and E^ . 

Property C.l. If "L^ is a diagonal matrix, the components of X are statistically 
independent, since p(x) can then be factored as a product of n scalar distribu- 
tions, according to (C.13). In other words, if the components of a Gaussian ran- 
dom vector are uncorrelated, they are statistically independent. 
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Property C.2. According to the properties of the variance (Sect. C.2.1), if the 
components of a Gaussian random vector X are uncorrected, thus independent, its 
covariance matrix Z^^ is positive definite. 

Property C.3. Every linear combination of Gaussian random vectors is still a 
Gaussian random vector. 



C.2.3 Vectorial Random Processes 

The concept of scalar random processes, X(t) , is easily extended to that of a ran- 
dom process with n components, where the X t (t) are scalar stochastic processes: 

x{t) = (x x {t) ... x n (t)) T . 

X(t) is called a vector-valued or vectorial stochastic process. 

One defines thus, similarly to the scalar case, the following notions, which will 
be presented briefly, first in the general case (non stationary case). 

Mathematical expectation, or mean of X(t) : 

e{x(0} = M0- 

Variance of X(t) : 

Var {X(0} = *xx if) = E { [X(t) - ^ (t)} [X(t) - yi x (*)] 
Autocovariance of the process X(?) : 

Cov{X(t 2 ),Xft)} = E xr (r 2 ,« 1 ) = E{[X(r 2 )-M'2)][Xtt)-M.jr(' 
Autocorrelation of the process X(t) : 

A xx (t 2 ,t l ) = E{x(t 2 )X J (t l )}. 



t)f 



Remark C.2. To simplify writings, when there is no ambiguity, in particular when 
only one stochastic process X(t) is involved, the indices XX are often omitted. 

In Table C.4 are collected the main definitions and properties of vectorial sto- 
chastic processes, both in continuous and discrete time. In this second case, an ad- 
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ditional simplification of the writings consists in using indices, instead of paren- 
theses, for the discrete time steps, each time there is no ambiguity. 

Table C.4 Main concepts of continuous and discrete vectorial random processes. 
Continuous case Discrete case 

X(0 = Var{X(0} ZOt) = Z,=Var{Xj (C.14) 

r(/ 2 ,^) = Cov{X(/ 2 ),Xft)} Iij,k) = -L jk =Cov{X ; ,X,} 

1. Properties: 

I(M) = Var{X(0} Z(k,k) = 2, ,, = Var{X,} (C.15) 

ljt 1 ,t I ) = Z r (t 2 ,t 1 ) ^,j=^L 

2. Stationary processes: 

t 2 -t 1 =T j-k = m 

v-xtt) = Vx; £(*>0 = 2 i»iW = v-x> ^k,k = 2 

Autocovariance matrix: 

l(t 2 ,t l ) = I^T) = Cov{X(t + T),X(t)} 2 7>t =2,„=Cov{X t+ra ,Xj (C16) 

N.B.: the order is important: 

Z(-r) = Z T (r) 3U,=l£ (C.17) 

Autocorrelation matrix: 

Mh,h) = A(r) = E{x(? + r)X T (0} A ;U = A,„ = e{x, + ,„X^} 

3. Centered stationary processes (\i x = 0) : 

Z(t) = A(t) £„,=a,„ 

autocovariance matrix = autocorrelation matrix 

4. Power spectral density: 

*(w)=f A(r) e-^rfr O(w) = ^ A(£) e~ Jku 



k=- 



(C.18) 

A(t) = — f ®(cu)e>-" T dcu A(k) = — (* <b(uj)e ik "duj (C.19) 

Note that A(r) , respectively A(£) , can be replaced by Z(r) , respectively 
£(&) = S t , in (C.18) and (C.19) in the centered case. 
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C.2.4 Gaussian Process (Scalar and Vectorial Cases) 

A scalar stochastic process X(t) is called Gaussian if, for any sequence of in- 
stants t\, t 2 , ..., t h and whatever i, the random vector whose components are snap- 
shots of the stochastic process taken at these instants, 

S = (X(t x ) X{t 2 ) ■■■ X(t { )) , has a Gaussian distribution, as given by (C.13) 
with X replaced by S . 

The same definition applies of course also to a vectorial Gaussian stochastic proc- 
ess X(t) : it is a stochastic process for which any sequence of random vectors 

X(7j ),X(t 2 ),... ,X(t t ) has a Gaussian joint distribution. 

The particularity of a Gaussian process X(7) is that it is completely determined by 
the knowledge of its two first moments, E{X(?)} and Cov{X(?[),X(? 2 )} . 



C.2. 5 Discrete White Noise (Vectorial Case) 

Definition C.5. A discrete white noise is a centered vectorial stochastic process 
V k , which consists of a sequence of uncorrelated random vectors. It is thus char- 
acterized by 

[E{VJ = 0, Vk 

, - (C20) 

|Cov{V„V ; } = QA y 

where Q k denotes a square, symmetric, positive semidefinite matrix 
(see Sect. C.2.1, Properties), which becomes a constant matrix Q if the white 
noise is stationary. 
If the white noise is in addition Gaussian, the random vectors V^ are independent, 

and consequently Q k is positive definite (see Sect. C.2. 2, Property C.2). 



C.2.6 Continuous White Noise (Vectorial Case) 

Definition C.6. A continuous white noise is a centered vectorial stochastic proc- 
ess V(t) , which consists of uncorrelated random vectors V(t) , V(t ! ) , whatever 

t^t' At is thus characterized by 



332 C Review of Probability Theory 

E{V(0} = 0, W 

r , - (C21) 

Coy {V(t),V(t')} = Q(t) b(t -t') 

where Q(t) denotes a square, symmetric, positive semidefinite matrix 
(see Sect. C.2.1, properties), which becomes a constant matrix Q if the white 
noise is stationary. 

If the white noise is in addition Gaussian, the random vectors V(t) and V(t') , 
with t^t' , are independent, and consequently Q(t) is positive definite 
(see Sect. C.2.2, Property C.2). 

Spectral property. With (C.18) we deduce from (C.21) that the power spectral 
density of stationary continuous white noise is constant, and equal to Q . 

Remark C.3. The above definitions C.5 and C.6 hold of course also in the scalar 
situation. 



D Simulation Diagrams 



D.l Simulink® Universal Simulation Diagrams 

The diagrams in Fig. D.l and Fig. D.2 for continuous-time and discrete-time simu- 
lations are of a universal nature, in the sense that they are adequate for all the ex- 
ercises proposed in the book. They comprise a state-feedback controller, an ob- 
server in its generalized form, a gain compensation (feedforward) matrix and error 
integrators. The parameters the user will have to adjust are listed in Table D.l. In 
the two diagrams, the C parameter of the State-Space block simulating the plant is 
an identity matrix, of suitable dimensions, in order to give access to the full state 
vector at the output of this block. The measurement matrix C m which builds the 

real plant output is introduced separately in the diagram. The selection matrix S,- 
is used for the feedback with integral action; the matrix S c is used in conjunction 
with the gain compensation (or anticipation) term M (for more details, see 
Chap. l,Sect. 1.8.3 and 1.1.3). 

Manual switches enable the user to disable unused functions, such as e.g. the 
observer (switch S 5 ) in the case that the simulation of a direct state feedback is 

desired, this state being assumed entirely accessible. If no observer has been cal- 
culated yet at this point, the corresponding block parameters are set to zero by the 
program. 

The Sj switch gives the choice between a step reference and a pulse generator. 
It is to the user to set, in the first case the amplitude parameter {final value) y r 

and the application time (step time) start _reference of the step function, in the 
second case the amplitude, period and pulse width of the generated signal, by re- 
specting the dimension of the involved quantities (scalars of vectors). For more 
details about the dimensions of the output signals of these two blocks, the reader is 
invited to consult the on-line documentation of MATLAB®. Switches S 3 and S 4 

enable applying constant load and measurement perturbations. It is again to the 
user to make appropriate choices for the amplitudes pjoad , respectively 
pjneasurements, and the application times start_p_load, respectively 
start _p jneasurements, of these perturbations, with the same remark as above as 
to their dimensions. In the case of the discrete-time diagram, the switches S6 and 
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D Simulation Diagrams 



S 7 enable applying random disturbances, which are useful for studying noisy sys- 
tems and optimal filtering. 

The error integrators are aimed at simulating state-feedback controls with inte- 
gral action, as described in Sect. 1.8. They can be disabled in this diagram, when 
not used, by choosing L : = L and L 2 = . 

The return to equilibrium without any input signal and from a non vanishing 
initial state x of the plant and/or an initial state z of the observer can be studied 

by swapping the switch S 2 with of course in this case L 2 = . 

In the various diagrams the user will have to adjust the number of channels 
{Number of axes) of the oscilloscopes, since this parameter can be introduced only 
numerically. After having checked the box Save data to workspace, he will also 
need to fill in the field Variable name which appears under the tab Data history 
with the corresponding name used in the subroutine Simulation JScopes.m in- 
cluded in mmce.zip. 

Although not essential, zero-order holds are inserted in front of the oscillo- 
scopes in the discrete-time diagram, in order to enforce the staircase aspect of the 
oscillograms, since otherwise the oscilloscopes would interpolate the plots be- 
tween measurement points. 



Table D.l Parameters of the simulation blocks. 



Continuous state-space model (plant): 

A: A B:[B E] C:eye(n) 
D: zeros(n, p+re) 
Initial conditions : xOl 



Discrete state-space model (plant): 

A: Phi B: [Gamma E] C: eye(n) 

D: zeros(n, p+re) 

Initial conditions: xOl; Sample time: Ts 



Generalized observer: 



A:F B:[J G] C:H 
D:[zeros(n,p) K] 
Initial conditions: zOl 



Generalized observer: 

A:F B:[J G] C:H 

D:[zeros(n,p) K] 

Initial conditions: [zO]; Sample time: Ts 



Step: Step time: startreference; Initial value: 0; Final value: yr; Sample time: 
Pulse generator: Amplitude: yr; Period: simulation_horizon/2.5; Pulse width: 

50% 
Load disturbance: Step time: start_p_load; Initial value: 0; Final value: pload 

(p_load*Ts in the discrete case) 
Measurement disturbance: Step time: start_p_measurements; Initial value: 0; 

Final value: pmeasurements 



Simulation / Configuration Parameters: 

Solver stop time: simulationhorizon 

Solver options: Variable-step (continuous); Fixed-step; Fixed step size: Ts 
(discrete) 



D. 1 Simulink® Universal Simulation Diagrams 
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Measurement 
S4 I . — , disturbance 




Estimated continuous 
state 



Fig. D.l Universal simulation diagram for continuous- time state- feedback control laws 
and observers {Universal _contimious_simulation.mdl). 



EH r^9 



H 




Fig. D.2 Universal simulation diagram for discrete-time state-feedback control laws 
and observers (Universal_discrete_simulation.mdl). 
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D.2 Specific Simulation Diagram for the Inverted 
Pendulum 



The third simulation diagram is on the contrary a specific discrete-time diagram, 
which serves only for the simulation of the inverted pendulum and takes into ac- 
count its nonlinearity and the compensation of it. It is used exclusively for the 
simulations of Exercise 2 of Chapter 2. 

A Coulomb & Viscous Friction block is used to model the dry friction. The Co- 
efficient of viscous friction of this block is set to zero. The switch S2 activates of 
disables this nonlinearity. 

The observer, of reduced order, is represented in this diagram in details by 
means of the various matrices involved in its generalized representation. The 
switch S3 commutes the dry friction compensation mode from fixed threshold to 
estimation by observer. The switch S 4 enables suppressing any compensation. 




Control Saturation 
(state feedback) 



Discrete 
measurements 



Comparison 

real-estimate 

(position and angle) 



Estimated discrete 
state 



Fig. D.3 Simulation diagram for the inverted pendulum, with disturbance observer 
{Pendulum _and_frictions.mdl). 
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dead-beat observer, 1 06 
decoupled control, 32 
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targeted choice, 50, 107 
elimination lemma, 270 
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variance minimization, 196 
estimation theory, 1 9 1 
extended Kalman filter, 244 

equations linearization, 245 
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Falb-Wolovich, 32, 65 
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finite settling-time, 7, 19, 21 
free response, 66, 75 
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functional optimization, 130 
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Hamilton's 
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principle, 138 
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definition, 133 
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inverted pendulum (Amira), 59, 62, 120, 
122, 181,286,336 
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Kalman canonical decomposition, 297 

Kalman duality, 1 06 

Kalman filter, 8, 79, 213, 218, 260 

deterministic term, 229 

estimator filter, 220 

extended see extended Kalman filter 

free wheel, 249 

gain matrix, 223 

in the loop, 236 

initialization, 220 

physical interpretation, 2 1 9 

predictor filter, 220 

state representation, 23 1 

stationary, 227, 229, 231, 243 

variance propagation equation, 225 
Kronecker product of matrices, 282 
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Lagrange multipliers, 132 

Leibnitz integral rule, 143 

limit cycle, 128 

Linear Matrix Inequality (LMI), 267 

LMI 

alpha-stabilization, 274, 276, 286 

detectability, 277 

detection by observer, 277 

inverse problem of optimal control, 
283,286,289 

LQC design, 278, 286 

matrix variable, 269, 270, 273 
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region, 282 
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solvers, 285, 290 

stability, 273 

stabilizability, 274 

state-feedback stabilization, 274 
LQC, 144, 173, 176, 181, 182, 186,286 

solved with LMIs, 278 
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Lyapunov, 267, 270 

function, 272, 278 

matrix, 281,285 

stability condition, 267, 272, 283 
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MATLAB 

notations correspondance, 243 



Index 



343 



solved exercises, 59, 64, 69, 71, 74, 

122, 180, 181, 185, 188,249, 
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matrix 

condition number, 3 1 6 

exponential, 298, 305 

inverse, 308 

inversion lemma, 308 

modal, 23, 45, 50 

partitioning, 89, 308 

positive definite (semidefinite), 317 

rank, 314 

selection, 57, 333 

square root, 318 

transfer, 30 
MIMO systems, 7, 18, 34, 46 
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minimal realization, 298 
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decoupling, 67, 69 
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input-output decoupling, 42 
modal control (simple), 7, 22, 26, 41, 65 

decoupling, 7 

design method, 22 

influence on other coordinates, 26 

input-output decoupling, 22, 23 

principle, 24 
modes 

separate control, 22 

unstable internal, 43 
Mullin and De Barbeyrac (algorithm), 20 
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noise 

covariance matrices, 232 
measurement, 204, 214, 222, 235, 249, 

251 
process (or state), 204, 214, 222, 232, 
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noisy systems, 78 
null space, 295, 297,298, 314 
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canonical decomposition, 297 
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criterion, 296 
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observable subspace, 297 
observer, 8, 78 

complete modal design, 1 07 

disturbance, 115 
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initial state, 112, 124 
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inverse problem, 283 
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output weighting, 254 

return difference, 164, 165, 167 
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optimal filtering, 8, 213 
optimality principle, 170 
order difference, 30 

maximal, 34 

of the output, 30, 33 
orthogonal complement, 270, 295, 314 
orthogonality principle, 197, 199 
output weighting, 145 
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parameter matrix, 1 06 
parameter vectors, 8 

choice, 38 

definition, 35 

in optimal control, 157 

invariance, 37 

targeted choice, 50, 75 
pole assignment see pole placement 
pole placement, 7, 9, 34, 62, 106, 281 
Pontryagin, 130 

Pontryagin's maximum principle, 139, 172 
power spectral density, 209, 233, 235, 258 
probability density function, 320 
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quadratic 

criterion, 131 
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random variables, 193, 319 
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covariance matrix, 326 
Gaussian, 328 
variance, 326 
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difference equation, 163, 220 
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154, 174,225,263 
discrete algebraic equation (DARE), 

163, 167 
matrix inequality, 270 
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loss of, 264 
Roppenecker's formula, 35, 36, 49, 50, 70, 
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Rosenbrock matrix, 69, 300 
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Schur lemma, 268, 270, 275, 279, 281, 282 
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selection matrix, 57, 333 

separation theorem, 110, 111 

shopping cart, 1 82 

similitude transformation, 301 

simulation diagrams, 335, 336 

singular value decomposition, 295, 298, 

315 
singular vectors, 315 
SISO systems, 7, 18,34,51 
slack variable, 269 
stabilizability, 294 
state 

equations (Laplace or z transform), 27, 
300 
state feedback, 2, 15, 18, 20, 77 
state reconstructor see observer 
state space, 20 

control, 1 

representation, 26, 36, 292, 334 
static error, 54 
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steady state, 4, 6 
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gain, 66 
stochastic difference equation, 200 
stochastic differential equation, 207, 210 
stochastic processes, 322 
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Gaussian, 259, 331 

generalized model, 204, 212 
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power spectral density (PSD), 324 

state model, 203,211 

stationary, 205, 212, 323 

variance evolution equation, 2 1 1 

variance propagation equation, 203 

vectorial, 329 
stochastic separation theorem, 260 
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Taylor series expansion, 245, 246 
three-tank system, 64, 65, 69, 71 
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